I would like to run an AWS lambda function every five minutes. In the AWS Management Console this is easy to set up, under the lambda function's "Event Sources" tab, but how do I set it up with Terraform?
I tried to use an aws_lambda_event_source_mapping resource, but it turns out that the API it uses only supports events from Kinesis and DynamoDB. When I try to use it with a scheduled event source, creation times out.
You can use an aws_cloudwatch_event_target resource to tie the scheduled event source (event rule) to your lambda function. You need to grant it permission to invoke your lambda function; you can use an aws_lambda_permission resource for this.
Example:
resource "aws_lambda_function" "check_foo" {
filename = "check_foo.zip"
function_name = "checkFoo"
role = "arn:aws:iam::424242:role/something"
handler = "index.handler"
}
resource "aws_cloudwatch_event_rule" "every_five_minutes" {
name = "every-five-minutes"
description = "Fires every five minutes"
schedule_expression = "rate(5 minutes)"
}
resource "aws_cloudwatch_event_target" "check_foo_every_five_minutes" {
rule = aws_cloudwatch_event_rule.every_five_minutes.name
target_id = "check_foo"
arn = aws_lambda_function.check_foo.arn
}
resource "aws_lambda_permission" "allow_cloudwatch_to_call_check_foo" {
statement_id = "AllowExecutionFromCloudWatch"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.check_foo.function_name
principal = "events.amazonaws.com"
source_arn = aws_cloudwatch_event_rule.every_five_minutes.arn
}
Verbjorns Ljosa's answer only includes permissions for cloudwatch to invoke the lambda. Have you specified the proper policy and iam role that allows the lambda to perform its actions?
resource "aws_iam_role" "check_foo_role" {
name="check-foo-assume-role"
assume_role_policy="assume_role_policy.json"
}
with assume_role_policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
and a policy referencing the above resource iam role I.e. something like
resource "iam_role_policy" "check-foo-policy" {
name="check-foo-lambda-policy"
# referencing the iam role above
role="${aws_iam_role.check_foo_role.id}"
policy="check-foo-policy.json"
}
and finally the json specifying the policy, check-foo-policy.json.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": ["*"]
},
{
"Effect": "Allow",
"Action": [
"abc:SomeAction",
"abc:AnotherAction",
],
"Resource": "some-arn-matching-the-actions"
}
Do note that you cannot specify a Resource restriction for the logs-related actions. abc:SomeAction might be ssm:GetParameter with an accompanying resource arn like "arn:aws:ssm:us-east-1:${your-aws-account-id}:parameter/some/parameter/path/*
As an addition to the accepted answer. Often times one would want the zip-file for the lambda to be created by terraform as well. To do so one can use the archive_file data source:
data "archive_file" "lambda_zip" {
type = "zip"
source_dir = "src"
output_path = "check_foo.zip"
}
resource "aws_lambda_function" "check_foo" {
filename = "check_foo.zip"
function_name = "checkFoo"
role = "arn:aws:iam::424242:role/something"
handler = "index.handler"
}
# then the rest from the accepted answer to trigger this
That is particularly helpful if the code is under version control, because then you can add the check_foo.zip to the .gitignore and there can never be a missmatch between the zip file and the source code that it is based on.
Related
I am trying write the logs of a lambda function into a CloudWatch Log Group created by terraform.
This is the lambda policy json -
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1580216411252",
"Action": [
"logs:CreateLogStream",
"logs:CreateLogDelivery",
"logs:PutLogEvents"
],
"Effect": "Allow",
"Resource": "arn:aws:logs:*:*:*"
}
]
}
This is the lambda assume policy json -
{
"Version": "2012-10-17",
"Statement": [{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}]
}
I have added this to the lambda.tf file -
resource "aws_cloudwatch_log_group" "example" {
name = "/test/logs/${var.lambda_function_name}"
}
Although the CloudWatch Log Group '/test/logs/${var.lambda_function_name}' is getting created through terraform, I am unable to write the log of the lambda function to this group.
If I change the lambda policy json to this -
{
"Version": "2012-10-17",
"Statement": [{
"Sid": "Stmt1580204738067",
"Action": "logs:*",
"Effect": "Allow",
"Resource": "*"
}]
}
Then It automatically stores the log in /aws/lambda/ directory.
How can I make sure that the lambda logs get written into a CloudWatch Log Group that I create and not in the /aws/lambda/ group created by lambda itself?
If you want Terraform to manage the CloudWatch log group, you have to create the log group ahead of time with the exact name the Lambda function is going to use for its log group. You can't change the name at all. Then in your Terraform you need to make the log group a dependency of the Lambda function, to make sure Terraform has a chance to create the log group before Lambda creates it automatically.
Just adding the log group as a dependency to the lambda is not enough. You also have to attach the IAM policy to the lambda role.
The steps are following:
Define the IAM role for lambda:
resource "aws_iam_role" "iam_for_lambda" {
name = "iam_for_lambda"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Action": "sts:AssumeRole"
}]
}
EOF
}
Define the IAM policy that allows lambda to create log streams and put log events
resource "aws_iam_policy" "function_logging_policy" {
name = "function-logging-policy"
policy = jsonencode({
"Version" : "2012-10-17",
"Statement" : [
{
Action : [
"logs:CreateLogStream",
"logs:PutLogEvents"
],
Effect : "Allow",
Resource : "arn:aws:logs:*:*:*"
}
]
})
}
Attach the policy to the IAM role created in step 1, by creating new resource 'aws_iam_role_policy_attachment'
resource "aws_iam_role_policy_attachment" "function_logging_policy_attachment" {
role = aws_iam_role.iam_for_lambda.id
policy_arn = aws_iam_policy.function_logging_policy.arn
}
Define the log group
resource "aws_cloudwatch_log_group" "lambda_log_group" {
name = "/aws/lambda/${var.lambda.function_name}"
retention_in_days = 7
lifecycle {
prevent_destroy = false
}
}
Define your lambda function with the depends_on parameter:
resource "aws_lambda_function" "lambda_function" {
filename = "../${var.lambda.function_filename}"
function_name = "${var.lambda.function_name}"
role = aws_iam_role.iam_for_lambda.arn
handler = "${var.lambda.handler}"
layers = [aws_lambda_layer_version.lambda_layer.arn]
depends_on = [aws_cloudwatch_log_group.lambda_log_group]
source_code_hash = filebase64sha256("../${var.lambda.function_filename}")
runtime = "python3.9"
}
The IAM policy creation & attachment comes from this article, the rest is from my personal project that worked for me.
I am learning Terraform. I am trying to create a new Lambda function. And I realized that I also need to create an IAM role. So I am trying to do both using Terraform. But it does not allow me to create the role.
This is my Terraform file
provider "aws" {
profile = "default"
region = "eu-west-1"
}
data "aws_iam_policy" "AWSLambdaBasicExecutionRole" {
arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}
resource "aws_iam_role" "terraform_function_role" {
name = "terraform_function_role"
assume_role_policy = "${data.aws_iam_policy.AWSLambdaBasicExecutionRole.policy}"
}
resource "aws_lambda_function" "terraform_function" {
filename = "terraform_function.zip"
function_name = "terraform_function"
handler = "index.handler"
role = "${aws_iam_role.terraform_function_role.id}"
runtime = "nodejs8.10"
source_code_hash = "${filebase64sha256("terraform_function.zip")}"
}
This is the error that I am getting
Error creating IAM Role terraform_function_role: MalformedPolicyDocument: Has prohibited field Resource
status code: 400
How do I fix this?
IAM Role's trust relationship (or assume role policy) defines which resource / service can assume the role. In this, we don't define the Resource field. Hence we can't attach IAM policies or use that policy as is. The correct format for Trust Relationship is:
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Action": "sts:AssumeRole"
}]
}
In this scenario, all Lambda functions in your account can assume this role.
You can refer this AWS link for more examples.
Edit: Based on #ydaetskcoR comment, here's a working example:
provider "aws" {
profile = "default"
region = "eu-west-1"
}
data "aws_iam_policy_document" "AWSLambdaTrustPolicy" {
statement {
actions = ["sts:AssumeRole"]
effect = "Allow"
principals {
type = "Service"
identifiers = ["lambda.amazonaws.com"]
}
}
}
resource "aws_iam_role" "terraform_function_role" {
name = "terraform_function_role"
assume_role_policy = "${data.aws_iam_policy_document.AWSLambdaTrustPolicy.json}"
}
resource "aws_iam_role_policy_attachment" "terraform_lambda_policy" {
role = "${aws_iam_role.terraform_function_role.name}"
policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}
resource "aws_lambda_function" "terraform_function" {
filename = "terraform_function.zip"
function_name = "terraform_function"
handler = "index.handler"
role = "${aws_iam_role.terraform_function_role.arn}"
runtime = "nodejs8.10"
source_code_hash = "${filebase64sha256("terraform_function.zip")}"
}
The changes from your code include the following:
Updated aws_iam_policy_document resource for assume role permissions
Changed aws_iam_role resource to use the above mentioned policy document
Created aws_iam_role_policy_attachment to attach LambdaBasicExecution policy (which enables logging to CloudWatch)
Updated aws_lambda_function resource to use IAM Role's ARN instead of Id because Lambda function needs the ARN
Since you are still in the learning phase, I suggest you move to terraform 0.12 instead, so you can use things like templatefile. therefore you don't need to create data objects.
One other thing is to always use the Least Privilege Principle when creating policies, meaning your Resource (Lambda, on this case) will only have access to what it needs. For now, it's only CloudWatch, but in a real world scenario this is very likely not the case.
Back to your question, here's how you can create an IAM Role, an IAM Policy and finally an IAM Policy Attachment (this is the bridge between the policy and the role) as well as the AssumeRolePolicy (this is the trust relationship between the service it's going to use it and the role itself). I have extracted it all into templates for you so it's easier to maintain later on. The gist (for an easier read on the eyes) can be found here.
# iam_role
resource "aws_iam_role" "iam_role" {
name = "iam-role"
assume_role_policy = templatefile("${path.module}/templates/lambda-base-policy.tpl", {})
}
#content of lambda-base-policy.tpl
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
#iam_policy
resource "aws_iam_policy" "policy" {
name = "policy"
policy = templatefile("${path.module}/templates/cloudwatch-policy.tpl", {})
}
#content of cloudwatch-policy.tpl
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "*"
}
]
}
#iam_policy_attachment
resource "aws_iam_policy_attachment" "policy_attachment" {
name = "attachment"
roles = ["${aws_iam_role.iam_role.name}"]
policy_arn = "${aws_iam_policy.policy.arn}"
}
As mentioned in the comment, you have to create assume role then attach the assume rule with your newly created policy, here is the complete working example.
#assume role
resource "aws_iam_role" "role" {
name = "test-alb-logs-to-elk"
path = "/"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
# Created Policy for IAM Role (s3 and log access)
resource "aws_iam_policy" "policy" {
name = "my-test-policy"
description = "A test policy"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:Get*",
"s3:List*"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"logs:*"
],
"Resource": "arn:aws:logs:*:*:*"
}
]
}
EOF
}
# Attached IAM Role and the new created Policy
resource "aws_iam_role_policy_attachment" "test-attach" {
role = "${aws_iam_role.role.name}"
policy_arn = "${aws_iam_policy.policy.arn}"
}
# Created AWS Lamdba Function: Memory Size, NodeJS version, handler, endpoint, doctype and environment settings
resource "aws_lambda_function" "elb_logs_to_elasticsearch" {
function_name = "mytest-alb-logs-to-elk"
description = "elb-logs-to-elasticsearch"
memory_size = 1024
filename = "terraform_function.zip"
runtime = "nodejs8.10"
role = "${aws_iam_role.role.arn}"
handler = "index.handler"
}
The goal here is to create scheduled snapshots of EBS volumes. Looking at Terraform's documentation for aws_cloudwatch_event_target it doesn't seem possible, but I could be missing something.
Cloudwatch Events built-in targets just seem to require an input parameter as well as the ARN that is shown for adding a message to an SNS queue in the example for aws_cloudwatch_event_rule or sending to a Kinesis stream in aws_cloudwatch_event_target.
So we should just be able to do something like this:
resource "aws_cloudwatch_event_target" "ebs_vol_a" {
target_id = "ebs_vol_a"
rule = "${aws_cloudwatch_event_rule.snap_ebs.name}"
arn = "arn:aws:automation:${var.region}:${var.account_id}:action/EBSCreateSnapshot/EBSCreateSnapshot_ebs_vol_a"
input = "\"arn:aws:ec2:${var.region}:${var.account_id}:volume/vol-${var.ebs_vol_a_id}\""
}
resource "aws_cloudwatch_event_rule" "snap_ebs" {
name = "snap-ebs-volumes"
description = "Snapshot EBS volumes"
schedule_expression = "rate(6 hours)"
}
I haven't yet tested this but it should work. Obviously you probably want to get the EBS volume IDs from the resource you created them but that's beyond the scope of the question. I've also guessed at the ARN after creating a rule in the AWS console and then looking at the output of aws events list-targets-by-rule where it seems to add the rule name to the ARN of the target but that may not always be true/necessary.
I was able to get this working entirely in terraform by tweaking this response provided by D Swartz. I had to modify the aws_cloudwatch_event_target resource in a few ways:
The arn field needs to point to the target/create-snapshot event instead of the action/EBSCreateSnapshot automation action.
The input field needs to be set to the desired volume's id rather than its arn.
The role_arn needs to be set to the arn of the aws_iam_role that will be running the event.
The updated aws_cloudwatch_event_target resource looks like this:
resource "aws_cloudwatch_event_target" "example_event_target" {
target_id = "example"
rule = "${aws_cloudwatch_event_rule.snapshot_example.name}"
arn = "arn:aws:events:${var.aws_region}:${var.account_id}:target/create-snapshot"
input = "${jsonencode("${aws_ebs_volume.example.id}")}"
role_arn = "${aws_iam_role.snapshot_permissions.arn}"
}
Full code snippet below:
resource "aws_cloudwatch_event_rule" "snapshot_example" {
name = "example-snapshot-volumes"
description = "Snapshot EBS volumes"
schedule_expression = "rate(24 hours)"
}
resource "aws_cloudwatch_event_target" "example_event_target" {
target_id = "example"
rule = "${aws_cloudwatch_event_rule.snapshot_example.name}"
arn = "arn:aws:events:${var.aws_region}:${var.account_id}:target/create-snapshot"
input = "${jsonencode("${aws_ebs_volume.example.id}")}"
role_arn = "${aws_iam_role.snapshot_permissions.arn}"
}
resource "aws_iam_role" "snapshot_permissions" {
name = "example"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "automation.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
resource "aws_iam_policy" "snapshot_policy" {
name = "example-snapshot-policy"
description = "grant ebs snapshot permissions to cloudwatch event rule"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:Describe*",
"ec2:RebootInstances",
"ec2:StopInstances",
"ec2:TerminateInstances",
"ec2:CreateSnapshot"
],
"Resource": "*"
}
]
}
EOF
}
resource "aws_iam_role_policy_attachment" "snapshot_policy_attach" {
role = "${aws_iam_role.snapshot_permissions.name}"
policy_arn = "${aws_iam_policy.snapshot_policy.arn}"
}
edit: I'm not entirely clear if this is directly supported by AWS or not - based on their documentation it seems like this may not be an intended feature:
Creating rules with built-in targets is supported only in the AWS Management Console.
The previous answer was enough to get everything except for the IAM permissions on the event targets (i.e. go into the console, edit the rule, and in "Step 2", for the "AWS Permissions" section, create a new role, etc). To get this working in terraform, I just added a few resources:
resource "aws_cloudwatch_event_rule" "snapshot_example" {
name = "example-snapshot-volumes"
description = "Snapshot EBS volumes"
schedule_expression = "rate(24 hours)"
}
resource "aws_cloudwatch_event_target" "example_event_target" {
target_id = "example"
rule = "${aws_cloudwatch_event_rule.snapshot_example.name}"
arn = "arn:aws:automation:${var.aws_region}:${var.account_id}:action/EBSCreateSnapshot/EBSCreateSnapshot_example-snapshot-volumes"
input = "${jsonencode("arn:aws:ec2:${var.aws_region}:${var.account_id}:volume/${aws_ebs_volume.example.id}")}"
}
resource "aws_iam_role" "snapshot_permissions" {
name = "example"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "automation.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
resource "aws_iam_policy" "snapshot_policy" {
name = "example-snapshot-policy"
description = "grant ebs snapshot permissions to cloudwatch event rule"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:Describe*",
"ec2:RebootInstances",
"ec2:StopInstances",
"ec2:TerminateInstances",
"ec2:CreateSnapshot"
],
"Resource": "*"
}
]
}
EOF
}
resource "aws_iam_role_policy_attachment" "snapshot_policy_attach" {
role = "${aws_iam_role.snapshot_permissions.name}"
policy_arn = "${aws_iam_policy.snapshot_policy.arn}"
}
I came up with this solution which is compatible with AWS CloudWatch Rules and Amazon EventBridge:
resource "aws_cloudwatch_event_rule" "volume_snapshot_rule" {
name = "ebs-volume-snapshot"
description = "Create an EBS volume snapshot every 6 hours"
schedule_expression = "rate(6 hours)"
}
resource "aws_cloudwatch_event_target" "volume_snapshot_target" {
target_id = "ebs-volume-snapshot-target"
rule = aws_cloudwatch_event_rule.volume_snapshot_rule.name
arn = "arn:aws:events:eu-central-1:${data.aws_caller_identity.current.account_id}:target/create-snapshot"
role_arn = "arn:aws:iam::${data.aws_caller_identity.current.account_id}:role/create-ebs-snapshot"
input = "\"${aws_ebs_volume.storage.id}\""
}
data "aws_caller_identity" "current" {}
And for the IAM role
resource "aws_iam_role" "create_ebs_snapshot" {
name = "create-ebs-snapshot"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = "sts:AssumeRole"
Effect = "Allow"
Sid = ""
Principal = {
Service = "events.amazonaws.com"
}
},
]
})
inline_policy {
name = "policy"
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = ["ec2:CreateSnapshot"]
Effect = "Allow"
Resource = "*"
},
]
})
}
}
I need to ship my cloudwatch logs to a log analysis service.
I've followed along with these articles here and here and got it working by hand, no worries.
Now I'm trying to automate all this with Terraform (roles/policies, security groups, cloudwatch log group, lambda, and triggering the lambda from the log group).
But I can't figure out how to use TF to configure AWS to trigger the lambda from the cloudwatch logs.
I can link the two TF resources together by hand by doing the following (in the Lambda web console UI):
go into the lambda function's "Triggers" section
click "Add Trigger"
select "cloudwatch logs" from the list of trigger types
select the log group I want to trigger the lambda
enter a filter name
leave the filter pattern empty (implying trigger on all log streams)
make sure "enable trigger" is selected
click the submit button
Once that's done, the lambda shows up on the cloudwatch logs console in the subscriptions column - displays as "Lambda (cloudwatch-sumologic-lambda)".
I tried to create the subscription with the following TF resource:
resource "aws_cloudwatch_log_subscription_filter" "cloudwatch-sumologic-lambda-subscription" {
name = "cloudwatch-sumologic-lambda-subscription"
role_arn = "${aws_iam_role.jordi-waf-cloudwatch-lambda-role.arn}"
log_group_name = "${aws_cloudwatch_log_group.jordi-waf-int-app-loggroup.name}"
filter_pattern = "logtype test"
destination_arn = "${aws_lambda_function.cloudwatch-sumologic-lambda.arn}"
}
But it fails with:
aws_cloudwatch_log_subscription_filter.cloudwatch-sumologic-lambda-subscription: InvalidParameterException: destinationArn for vendor lambda cannot be used with roleArn
I found this answer about setting up a similar thing for a scheduled event, but that doesn't seem to be equivalent to what the console actions I described above do (the console UI method doesn't create an event/rule that I can see).
Can someone give me a pointer on what I'm doing wrong please?
I had the aws_cloudwatch_log_subscription_filter resource defined incorrectly - you should not provide the role_arn argument in this situation.
You also need to add an aws_lambda_permission resource (with a depends_on relationship defined on the filter or TF may do it in the wrong order).
Note that the AWS lambda console UI adds the lambda permission for you invisibly, so beware that the aws_cloudwatch_log_subscription_filter will work without the permission resource if you happen to have done the same action before in the console UI.
The necessary TF config looks like this (the last two resources are the relevant ones for configuring the actual cloudwatch->lambda trigger):
// intended for application logs (access logs, modsec, etc.)
resource "aws_cloudwatch_log_group" "test-app-loggroup" {
name = "test-app"
retention_in_days = 90
}
resource "aws_security_group" "cloudwatch-sumologic-lambda-sg" {
name = "cloudwatch-sumologic-lambda-sg"
tags {
Name = "cloudwatch-sumologic-lambda-sg"
}
description = "Security group for lambda to move logs from CWL to SumoLogic"
vpc_id = "${aws_vpc.dev-vpc.id}"
}
resource "aws_security_group_rule" "https-egress-cloudwatch-sumologic-to-internet" {
type = "egress"
from_port = 443
to_port = 443
protocol = "tcp"
security_group_id = "${aws_security_group.cloudwatch-sumologic-lambda-sg.id}"
cidr_blocks = ["0.0.0.0/0"]
}
resource "aws_iam_role" "test-cloudwatch-lambda-role" {
name = "test-cloudwatch-lambda-role"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow"
}
]
}
EOF
}
resource "aws_iam_role_policy" "test-cloudwatch-lambda-policy" {
name = "test-cloudwatch-lambda-policy"
role = "${aws_iam_role.test-cloudwatch-lambda-role.id}"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "CopiedFromTemplateAWSLambdaVPCAccessExecutionRole1",
"Effect": "Allow",
"Action": [
"ec2:CreateNetworkInterface"
],
"Resource": "*"
},
{
"Sid": "CopiedFromTemplateAWSLambdaVPCAccessExecutionRole2",
"Effect": "Allow",
"Action": [
"ec2:DescribeNetworkInterfaces",
"ec2:DeleteNetworkInterface"
],
"Resource": "arn:aws:ec2:ap-southeast-2:${var.dev_vpc_account_id}:network-interface/*"
},
{
"Sid": "CopiedFromTemplateAWSLambdaBasicExecutionRole1",
"Effect": "Allow",
"Action": "logs:CreateLogGroup",
"Resource": "arn:aws:logs:ap-southeast-2:${var.dev_vpc_account_id}:*"
},
{
"Sid": "CopiedFromTemplateAWSLambdaBasicExecutionRole2",
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": [
"arn:aws:logs:ap-southeast-2:${var.dev_vpc_account_id}:log-group:/aws/lambda/*"
]
},
{
"Sid": "CopiedFromTemplateAWSLambdaAMIExecutionRole",
"Effect": "Allow",
"Action": [
"ec2:DescribeImages"
],
"Resource": "*"
}
]
}
EOF
}
resource "aws_lambda_function" "cloudwatch-sumologic-lambda" {
function_name = "cloudwatch-sumologic-lambda"
filename = "${var.lambda_dir}/cloudwatchSumologicLambda.zip"
source_code_hash = "${base64sha256(file("${var.lambda_dir}/cloudwatchSumologicLambda.zip"))}"
handler = "cloudwatchSumologic.handler"
role = "${aws_iam_role.test-cloudwatch-lambda-role.arn}"
memory_size = "128"
runtime = "nodejs4.3"
// set low because I'm concerned about cost-blowout in the case of mis-configuration
timeout = "15"
vpc_config = {
subnet_ids = ["${aws_subnet.dev-private-subnet.id}"]
security_group_ids = ["${aws_security_group.cloudwatch-sumologic-lambda-sg.id}"]
}
}
resource "aws_lambda_permission" "test-app-allow-cloudwatch" {
statement_id = "test-app-allow-cloudwatch"
action = "lambda:InvokeFunction"
function_name = "${aws_lambda_function.cloudwatch-sumologic-lambda.arn}"
principal = "logs.ap-southeast-2.amazonaws.com"
source_arn = "${aws_cloudwatch_log_group.test-app-loggroup.arn}"
}
resource "aws_cloudwatch_log_subscription_filter" "test-app-cloudwatch-sumologic-lambda-subscription" {
depends_on = ["aws_lambda_permission.test-app-allow-cloudwatch"]
name = "cloudwatch-sumologic-lambda-subscription"
log_group_name = "${aws_cloudwatch_log_group.test-app-loggroup.name}"
filter_pattern = ""
destination_arn = "${aws_lambda_function.cloudwatch-sumologic-lambda.arn}"
}
EDIT: Please note that the above TF code was written years ago, using version 0.11.x - it should still work but there may be better ways of doing things. Specifically, don't use an inline policy like this unless needed, use an aws_iam_policy_document instead - they're just way easier to maintain over time.
Working with Terraform v0.12.29 and AWS provider v3.1.0 I encountered a strange issue which cost me a few hours of debugging.
In order to save others some valuable time I'll share it as a complementary to the accepted answer.
The value of the cloudwatch log group arn:
${aws_cloudwatch_log_group.test-app-loggroup.arn}
Isn't being interpolated correctly - there is a missing ":*" at the end of the output.
This leads to the error below:
Error creating {the-calling-service}: InvalidCloudWatchLogsLogGroupArnException:
Check the log group ARN: {the-calling-service} can't validate it.
Adding a :* postfix solved the issue:
source_arn = "${aws_cloudwatch_log_group.test-app-loggroup.arn}:*" #<----Notice the :* postfix
I would like to run an AWS lambda function every five minutes. In the AWS Management Console this is easy to set up, under the lambda function's "Event Sources" tab, but how do I set it up with Terraform?
I tried to use an aws_lambda_event_source_mapping resource, but it turns out that the API it uses only supports events from Kinesis and DynamoDB. When I try to use it with a scheduled event source, creation times out.
You can use an aws_cloudwatch_event_target resource to tie the scheduled event source (event rule) to your lambda function. You need to grant it permission to invoke your lambda function; you can use an aws_lambda_permission resource for this.
Example:
resource "aws_lambda_function" "check_foo" {
filename = "check_foo.zip"
function_name = "checkFoo"
role = "arn:aws:iam::424242:role/something"
handler = "index.handler"
}
resource "aws_cloudwatch_event_rule" "every_five_minutes" {
name = "every-five-minutes"
description = "Fires every five minutes"
schedule_expression = "rate(5 minutes)"
}
resource "aws_cloudwatch_event_target" "check_foo_every_five_minutes" {
rule = aws_cloudwatch_event_rule.every_five_minutes.name
target_id = "check_foo"
arn = aws_lambda_function.check_foo.arn
}
resource "aws_lambda_permission" "allow_cloudwatch_to_call_check_foo" {
statement_id = "AllowExecutionFromCloudWatch"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.check_foo.function_name
principal = "events.amazonaws.com"
source_arn = aws_cloudwatch_event_rule.every_five_minutes.arn
}
Verbjorns Ljosa's answer only includes permissions for cloudwatch to invoke the lambda. Have you specified the proper policy and iam role that allows the lambda to perform its actions?
resource "aws_iam_role" "check_foo_role" {
name="check-foo-assume-role"
assume_role_policy="assume_role_policy.json"
}
with assume_role_policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
and a policy referencing the above resource iam role I.e. something like
resource "iam_role_policy" "check-foo-policy" {
name="check-foo-lambda-policy"
# referencing the iam role above
role="${aws_iam_role.check_foo_role.id}"
policy="check-foo-policy.json"
}
and finally the json specifying the policy, check-foo-policy.json.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": ["*"]
},
{
"Effect": "Allow",
"Action": [
"abc:SomeAction",
"abc:AnotherAction",
],
"Resource": "some-arn-matching-the-actions"
}
Do note that you cannot specify a Resource restriction for the logs-related actions. abc:SomeAction might be ssm:GetParameter with an accompanying resource arn like "arn:aws:ssm:us-east-1:${your-aws-account-id}:parameter/some/parameter/path/*
As an addition to the accepted answer. Often times one would want the zip-file for the lambda to be created by terraform as well. To do so one can use the archive_file data source:
data "archive_file" "lambda_zip" {
type = "zip"
source_dir = "src"
output_path = "check_foo.zip"
}
resource "aws_lambda_function" "check_foo" {
filename = "check_foo.zip"
function_name = "checkFoo"
role = "arn:aws:iam::424242:role/something"
handler = "index.handler"
}
# then the rest from the accepted answer to trigger this
That is particularly helpful if the code is under version control, because then you can add the check_foo.zip to the .gitignore and there can never be a missmatch between the zip file and the source code that it is based on.