I am not able to find terraform module to create AWS SNS Topic subscription. e.g: I used "terraform-aws-modules/sns/aws" to create SNS topic. Can someone point me to source module for subscription?
You should prefer resources to modules unless you have a complex use case involving many interacting resources in a common pattern.
Modules are best suited to standardizing common patterns. A module can be made for a single resource, but it's rarely worth the overhead.
Here is a worked example based on a real system. This creates an SNS topic and subscribes a Lambda function to it:
resource "aws_sns_topic" "kpis" {
name = var.sns_topic_name
}
resource "aws_sns_topic_subscription" "invoke_with_sns" {
topic_arn = aws_sns_topic.kpis.arn
protocol = "lambda"
endpoint = module.kpis.function_arn
}
resource "aws_lambda_permission" "allow_sns_invoke" {
statement_id = "AllowExecutionFromSNS"
action = "lambda:InvokeFunction"
function_name = module.lambda.function_name
principal = "sns.amazonaws.com"
source_arn = aws_sns_topic.kpis.arn
}
You can read more about aws_sns_topic_subscription here:
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic_subscription
Related
The problem is pretty simple but hard to figure it out. I have a lambda function with versioning enabled deployed via terraform. The lambda allows the API Gateway to call into it via the below Terraform resource
resource "aws_lambda_permission" "my_lambda_permission" {
statement_id = "AllowExecutionFromAPIGatewayVersion${aws_lambda_function.mylambda.version}"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.mylambda.function_name
principal = "apigateway.amazonaws.com"
source_arn = "${aws_api_gateway_rest_api.apigw.execution_arn}/*/POST/redacted/path/to/resource"
qualifier = aws_lambda_function.myfunction.version
}
This is all fun and games and works for the current version. If I make a change to the lambda code and try to redeploy, the newer version gets the above policy and the previous version - the one that just worked - doesn't have a resource policy anymore.
How can I retain lambda previous versions' Resource-based policy?
I am trying to use Terraform to connect two AWS lambda functions to an S3 event.
My configuration looks like this:
resource "aws_s3_bucket_notification" "collection_write" {
bucket = aws_s3_bucket.fruit.id
lambda_function {
lambda_function_arn = aws_lambda_function.apple.arn
events = [ "s3:ObjectCreated:*" ]
filter_prefix = "foo/"
}
lambda_function {
lambda_function_arn = aws_lambda_function.banana.arn
events = [ "s3:ObjectCreated:*" ]
filter_prefix = "foo/"
}
}
But AWS / Terraform doesn't like this:
Error: Error putting S3 notification configuration: InvalidArgument: Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.
status code: 400
How should I write this in Terraform?
your terraform is not wrong it is just that s3 is limited to a single event notification. It is better to have the s3 event sent to an SNS topic which then triggers the lambdas to achieve the same functionality.
You can also achieve triggering multiple Lambda functions via AWS SQS. AWS SQS is powerful and easy to use Messaging queue for such use cases.
If I manually add an Integration Request of type Lambda function, an Api Gateway trigger is automatically added to the lambda function.
If I do it via Terraform, everything looks correct but when I go look at the Lambda function it has no trigger.
If I then manually update the Integration Request (change to Mock and back to Lambda Function) the trigger is added to the Lambda function? Everything works after that.
What am I missing?
resource "aws_api_gateway_integration" "integration" {
count = var.lambda_definition.apigateway ? 1 : 0
rest_api_id = "${data.terraform_remote_state.apigateway.outputs.apigateway_id}"
resource_id = aws_api_gateway_resource.api_proxy_resource[count.index].id
http_method = "${aws_api_gateway_method.method[count.index].http_method}"
integration_http_method = "ANY"
type = "AWS_PROXY"
uri = aws_lambda_function.lambda.invoke_arn
}
Since you've not mentioned whether you specified proper permissions for your function, my guess is that you are missing aws_lambda_permission. This will explicitly give permissions for the api to invoke your function.
The resource would be (example only):
resource "aws_lambda_permission" "allow_api" {
statement_id = "AllowAPIgatewayInvokation"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.lambda.invoke_arn
principal = "apigateway.amazonaws.com"
}
When you do it manually in console, the AWS setups all these permissions in the background.
Make sure that integration_http_method is set to POST and not to ANY as in your sample:
integration_http_method = "POST"
See AWS Docs - midway - red box that says '! Important':
For Lambda integrations, you must use the HTTP method of POST for the integration request, according to the specification of the Lambda service action for function invocations. The IAM role of apigAwsProxyRole must have policies allowing the apigateway service to invoke Lambda functions. For more information about IAM permissions, see API Gateway permissions model for invoking an API.
I have few SQS events in the S3 bucket notification. When i run the operation from Terraform those SQS events are lost as they are not part of state file. I cant use cli to import the events as the terraform will be run several time and it is not a good idea to import all the events everytime after terraform has completed the execution.
I am creating a S3 event from terraform and having a lifecycle to ignore SQS type :
resource "aws_s3_bucket_notification" "lambda_notification" {
bucket = "bucket1"
lambda_function {
lambda_function_arn = "function_arn"
events = ["s3:ObjectCreated:*"]
filter_prefix = "staging/inbound/Source_Contact/"
}
lifecycle {
ignore_changes = [
"SQS"
]
}
}
I want to know if lifecycle can be used to keep the SQS event
SQS events to Lambda are consumed on delivery unless they fail and you have a dead letter queue (DLQ) to collect them.
lifecycle ignore_changes won't be effective for your use regardless of what you set it to. It doesn't affect content in services, it affects how Terraform deploys resources when it detects changes to the resources versus your module source:
ignore_changes (list of attribute names) - By default, Terraform detects any difference in the current settings of a real infrastructure object and plans to update the remote object to match configuration.
In some rare cases, settings of a remote object are modified by processes outside of Terraform, which Terraform would then attempt to "fix" on the next run. In order to make Terraform share management responsibilities of a single object with a separate process, the ignore_changes meta-argument specifies resource attributes that
Terraform should ignore when planning updates to the associated remote object.
I tried the following and was able to retain the SQS events created out of tfe and create Lambda events from tfe.
resource "aws_s3_bucket_notification" "lambda_notification" {
bucket = "bucket1"
lambda_function {
lambda_function_arn = "function_arn"
events = ["s3:ObjectCreated:*"]
filter_prefix = "staging/inbound/Source_Contact/"
}
lifecycle {
ignore_changes = [
"queue."
]
}
}
Amazon released HTTP APIs for their api gateway product fairly recently, and I'd like to use Terraform to spin some up. I saw that Terraform has documentation for the aws_apigatewayv2_api resource, which will create the gateway itself, but I can't find documentation for routes, stages, or any other types of resources that I'll be needing in my gateway.
Are HTTP APIs just not fully supported by Terraform yet, or am I missing some documentation somewhere?
Update:
Per the closing comment on issue #11148, the AWS API Gateway HTTP API resources were first generally supported in terraform-aws-provider starting in v3.3.0. Later releases have already added new functionality.
Historical answer:
Indeed, it looks like the Terraform AWS provider does not yet support all of the resources necessary for an API Gateway HTTP API.
The aws_apigatewayv2_api resource was released on March 12, 2020 in terraform-provider-aws 2.53.0, but implementations of resources such as aws_apigatewayv2_route, aws_apigatewayv2_stage, and aws_apigatewayv2_integration have yet to be merged.
terraform-provider-aws issue #11148 has links to pending pull requests for the new resources.
Everything you need to use http v2 apis is now available in terraform. There is a good tutorial on creating them here.
Essentially, for a simple gateway+lambda combo you want something like this:
# Lambda function
resource "aws_lambda_function" "lambda" {
# ...
}
# HTTP API
resource "aws_apigatewayv2_api" "api" {
name = "api-${random_id.id.hex}"
protocol_type = "HTTP"
target = aws_lambda_function.lambda.arn
}
# Permission
resource "aws_lambda_permission" "apigw" {
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.lambda.arn
principal = "apigateway.amazonaws.com"
source_arn = "${aws_apigatewayv2_api.api.execution_arn}/*/*"
}