I am having some trouble writing the cloudformation script for cloudwatch event trigger to kick off my lambda script, I know I can do it through the console but my requirement is that I need to provision everything in cloudformation. I followed the documentation and it still haven't worked for me and I kept getting the error:
Template contains errors.: Invalid template property or properties
[rPermissionForEventsToInvokeLambda, rLambdaScheduledRule]
can someone point out what is the issue with this part of my cloudformation script? I followed the document almost to the letter and still having error, even the example in the documentation have the same error when I tried to validate it. my cloudformation code is below, any help is appreciated!
rLambdaScheduledRule:
Type: AWS::Events::Rule
Properties:
ScheduleExpression: rate(1 hour)
State: ENABLED
Targets:
Ref:
Fn::ImportValue:
Fn::Sub: rUploadLambda
Action: lambda:InvokeFunction
rPermissionForEventsToInvokeLambda:
Type: AWS::Lambda::Permission
Properties:
FunctionName:
Ref:
Fn::ImportValue:
Fn::Sub: rUploadLambda
Action: lambda:InvokeFunction
Principal: events.amazonaws.com
SourceArn:
Fn::GetAtt:
- rLambdaScheduledRule
- Arn
1) You must export the Lambda function ARN in the CloudFormation template in which you create the lambda function. You need to pass the Lambda function ARN as input to the cloudwatch event (The AWS::Events::Rule Targets attribute requires a resource ARN).
See a sample script below:
Resources:
# Create Controlled Lambda Function
myLambda:
Type: "AWS::Lambda::Function"
Properties:
Code:
S3Bucket: "lambda-bucket"
S3Key: "myhandler.zip"
Description: "Lambda handler"
FunctionName: "myhandler"
Handler: myhandler.myhandler
MemorySize: 128
Role: "arn:aws:iam::xxxxxxxxxxx:role/myLambdaExecutionRole-NC7FA7TUSZ5B"
Runtime: "python3.6"
Timeout: 10
# Output of the cf template
Outputs:
myLambdaArn:
Description: Arn of the my_lambda_function
Value: !GetAtt myLambda.Arn
Export:
Name: !Sub "${AWS::StackName}-LambdaArn"
2) When you create the CloudWatch Event, you need to pass the ARN of the lambda function created in Step1 as the Target.
See a sample script below:
Resources:
# Cloudwatch event to trigger lambda periodically
rLambdaScheduledRule:
Type: "AWS::Events::Rule"
Properties:
Description: "CloudWatch Event to trigger lambda fn"
ScheduleExpression: "rate(1 hour)"
State: "ENABLED"
Targets:
-
Arn:
Fn::ImportValue:
!Sub "${NetworkStackName}-LambdaArn"
Id: "targetevent_v1"
PermissionForEventsToInvokeLambda:
Type: "AWS::Lambda::Permission"
Properties:
FunctionName:
Fn::ImportValue:
!Sub "${NetworkStackName}-LambdaArn"
Action: "lambda:InvokeFunction"
Principal: "events.amazonaws.com"
SourceArn:
Fn::GetAtt:
- rLambdaScheduledRule
- Arn
The value of ${NetworkStackName} should be the StackName from Step1.
Some of the issues you need to correct in your template:
correct the Targets property of resource rLambdaScheduledRule.
remove Action property from resource rLambdaScheduledRule.
correct the FunctionName property of resource rPermissionForEventsToInvokeLambda.
Keeping above sample as reference, correct your template and try again.
Related
I am trying to set up a demo environment to try out SQS as an AWS Event Bridge Source. I tried uploading few documents to SQS to see if Event Bridge detects any change, but I don't see any events triggered. How can I test SQS as a source with AWS Event Bridge?
Resources:
Queue:
Type: AWS::SQS::Queue
Properties:
QueueName: !Sub ${AWS::StackName}
LambdaHandlerExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
EventConsumerFunction:
Type: AWS::Lambda::Function
Properties:
Handler: index.lambda_handler
Role: !GetAtt LambdaHandlerExecutionRole.Arn
Code:
ZipFile: |
import json
def lambda_handler(event, context):
print("Received event: " + json.dumps(event, indent=2))
Runtime: python3.7
Timeout: 50
EventRule:
Type: AWS::Events::Rule
Properties:
Description: eventEventRule
State: ENABLED
EventPattern:
source:
- aws.sqs
resources:
- !GetAtt Queue.Arn
Targets:
- Arn: !GetAtt EventConsumerFunction.Arn
Id: EventConsumerFunctionTarget
PermissionForEventsToInvokeLambda:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !Ref EventConsumerFunction
Action: lambda:InvokeFunction
Principal: events.amazonaws.com
SourceArn: !GetAtt EventRule.Arn
SQS data events (publishing new message) are not source events for Event Bridge (EB). Only management events can be picked up by EB, e.g.:
purging of the queue
creating of new queue
deletion of a queue
Also your event rule should be more generic for that:
EventRule:
Type: AWS::Events::Rule
Properties:
Description: eventEventRule
State: ENABLED
EventPattern:
source:
- aws.sqs
# resources:
# - !GetAtt Queue.Arn
Targets:
- Arn: !GetAtt EventConsumerFunction.Arn
Id: EventConsumerFunctionTarget
You can also enable CloudWatch trial and detect API events for the SQS. This should enable fetching more events.
I might be late but this can benefit someone else,
have a look at this:
https://docs.aws.amazon.com/cdk/api/latest/docs/#aws-cdk_aws-ecs-patterns.QueueProcessingFargateService.html
This will handle scaling of the Fargate container based on a number of messages in the SQS Queue.
a simplest stack can be defined using AWS CDK as following:
queue = sqs.Queue(stack, "Queue")
cluster = aws_ecs.Cluster(
stack, 'FargateCluster'
)
queue_processing_fargate_service = QueueProcessingFargateService(stack, "Service",
cluster=cluster,
memory_limit_mi_b=512,
image=ecs.ContainerImage.from_registry("test"),
command=["-c", "4", "amazon.com"],
enable_logging=False,
desired_task_count=2,
environment={
"TEST_ENVIRONMENT_VARIABLE1": "test environment variable 1 value",
"TEST_ENVIRONMENT_VARIABLE2": "test environment variable 2 value"
},
queue=queue,
max_scaling_capacity=5,
container_name="test"
)
Within AWS I've created a CloudTrail which is then filtered by an Eventbridge Rule to only look for certain events within CloudTrail that correspond to resources being created on AWS. Reason being is that I've created some code for Lambda that will automatically tag resources dependent on events passed to it from Eventbridge. When I manually connect eventbridge rule & lambda together it all works fine. However, when I deploy my stack using Cloudformation my Lambda doesn't show Eventbridge as an eventsource/trigger for the function and I don't know why. Below is my Cloudformation template alongside what is shown on AWS Lambda vs what I expect to be seen.
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
# ---------------------------------------------------------------------------- #
# Input parameters #
# ---------------------------------------------------------------------------- #
Parameters:
ProjectName:
Type: String
Default: 'AutoTagger'
Description: ""
# ---------------------------------------------------------------------------- #
# Resources #
# ---------------------------------------------------------------------------- #
Resources:
AutoTaggerLambda:
Type: AWS::Serverless::Function
Name: auto-tagger-lambda
Properties:
CodeUri: release/archive.zip
Handler: auto-tagger/main.lambda_handler
Runtime: python3.9
Policies: [AWSLambda_FullAccess]
MemorySize: 128
Timeout: 30
Tags:
- Key: "project_name"
Value: !Ref ProjectName
TagEvents:
Type: "AWS::Events::Rule"
Properties:
Description: "Rule to trigger lambda"
Name: "TagEvents"
EventPattern: {
"detail-type": ["AWS API Call via CloudTrail"],
"detail": {
"eventSource": ["ec2.amazonaws.com", "rds.amazonaws.com", "lambda.amazonaws.com", "s3.amazonaws.com", "dynamodb.amazonaws.com", "elasticfilesystem.amazonaws.com"],
"eventName": ["CreateVolume", "RunInstances", "CreateImage", "CreateSnapshot", "CreateDBInstance", "CreateFunction20150331", "UpdateFunctionConfiguration20150331v2", "UpdateFunctionCode20150331v2", "CreateBucket", "CreateTable", "CreateMountTarget"]
}
}
State: "ENABLED"
Targets:
- Arn: !GetAtt AutoTaggerLambda.Arn
Id: "TagEventsTargetLambda"
Do I need to add an event to the lambda also? I'm a little confused.
You are missing permissions. An AWS::Lambda::Permission resource gives your EventBridge rule permission to invoke the Lambda. It is added to the Lambda's resource-based policy.
PermissionForEventsToInvokeLambda:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !GetAtt AutoTaggerLambda.Arn
Action: 'lambda:InvokeFunction'
Principal: 'events.amazonaws.com'
SourceArn: !GetAtt TagEvents.Arn
Hi I am using AWS SAM to deploy a serverless application that also relies on 1Click devices I have the template deploying fine but I have to exclude this value:
CallbackOverrides:
onClickCallback:
here is the template file
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: securitybutton
Globals:
Function:
Timeout: 10
Resources:
SecurityButtonFunction:
Type: AWS::Serverless::Function
Policies:
- AmazonSNSFullAccess
Properties:
CodeUri: src
Handler: index.lambdaHandler,
Runtime: nodejs12.x
MemorySize: 512
Events:
IoTRule:
Type: IoTRule
Properties:
Sql: SELECT * FROM 'iotbutton/*'
#TOPICS
BRSecurityButtonTopic:
Type: AWS::SNS::Topic
#IoT
#project
BRSecurityButtonProject:
Type: "AWS::IoT1Click::Project"
Properties:
ProjectName: "BRSecurityButton"
Description: "BRSecurityButton"
PlacementTemplate:
DefaultAttributes:
SNSTopic: Changeme
Location: Changeme
Theatre: Changeme
DeviceTemplates:
SecuityButtonTemplate:
DeviceType: "button"
CallbackOverrides:
onClickCallback: !GetAtt
- SecurityButtonFunction
- Arn
Outputs:
SecurityButtonFunction:
Description: "Security Button Lambda Function ARN"
Value: !GetAtt SecurityButtonFunction.Arn
Here is the resulting cloudFormation error:
Lambda function
arn:aws:lambda:us-east-1:LALALALALALA:function:securitybutton-prod-SecurityButtonFunction-6OB47JEIU192
cannot be invoked. (Service: AWSIoT1ClickProjects; Status Code: 400;
Error Code: InvalidRequestException; Request ID:
f0e94cd5-0310-4dcf-8d4a-a5ee8b102590; Proxy: null)
on the logical ID of BRSecurityButtonProject
any help would be appreciated
I found that the issue was related to IAM. Added this to the SAM template, which granted access for any IoT 1-click projects in the account to invoke the Lambda function.
SecurityButtonFunctionIoTPermission:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !GetAtt SecurityButtonFunction.Arn
Action: lambda:InvokeFunction
Principal: iot1click.amazonaws.com
SourceAccount: !Ref "AWS::AccountId"
Currently my serverless.yml file looks like this:
service: bbb
provider:
name: aws
runtime: go1.x
stage: dev
package:
exclude:
- ./**
include:
- ./bin/**
functions:
ccc:
handler: bin/executable
name: my1minutelambda
role:
'Fn::GetAtt':
- mylambdaexecutionrole
- Arn
resources:
Resources:
mylambdaexecutionrole:
Type: AWS::IAM::Role
Properties:
RoleName: my-basiclambdaexec-role
Description: This is my basiclambdaexecution role
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Principal:
Service:
- "lambda.amazonaws.com"
Action:
- "sts:AssumeRole"
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
myminschedulerevent:
Type: AWS::Events::Rule
Properties:
Description: This is my 1 minute rate scheduler.
Name: my-1-min-trigger-event-scheduler
ScheduleExpression: rate(1 hour)
Targets:
-
Arn: "arn:aws:lambda:us-east-1:111111111111:function:my1minutelambda" #update your a/c Id
Id: "TargetFunctionV1"
command used to deploy: sls deploy
After deployment finished, I can see on aws management console that all my resources got created.
BUT I am not able to see cloudwatch trigger extablishment for my lambda function.
See below screenshot:
CloudWatch Event Rule created successfully. (Target section pointing to my lambda function)
Trigger link not established for my lambda:
Please let me know what i am missing here. Thank you.
Update#1:
After adding following lines (as suggested by Marcin), I am able to see "CloudWatch event".
EventsPermission:
Type: AWS::Lambda::Permission
Properties:
FunctionName: my1minutelambda
Action: lambda:InvokeFunction
Principal: events.amazonaws.com
SourceAccount: !Ref 'AWS::AccountId'
SourceArn: !GetAtt myminschedulerevent.Arn
But, I can't see CloudWatch logs!! So, I can't findout if my lambda function is executing. Please see image below:
I tried to replicate the issue using serverless framework.
To do so I added the following AWS::Lambda::Permission to the end of your template:
EventsPermission:
Type: AWS::Lambda::Permission
Properties:
FunctionName: dsfgsdfg # <-- REPLACE this with your function name my1minutelambda
Action: lambda:InvokeFunction
Principal: events.amazonaws.com
SourceArn: !GetAtt myminschedulrevent.Arn
After adding the permissions, the console showed the trigger as expected:
If all you are trying to do is get a Lambda function to execute on a schedule, the Serverless Framework already includes an event type expressly for that purpose:
functions:
crawl:
handler: crawl
events:
- schedule: rate(2 hours)
- schedule: cron(0 12 * * ? *)
It will set everything up for you with no need to add additional CloudFormation. You can find the documentation here: https://www.serverless.com/framework/docs/providers/aws/events/schedule/#schedule/
ScheduledRule:
Type: AWS::Events::Rule
Properties:
Name: "SphEvent"
Description: "ScheduledRule"
ScheduleExpression: "rate(1 hour)"
State: "ENABLED"
Targets:
- Arn: !Sub "arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:lambda-name"
Id: "TargetFunctionV1"
PermissionForEventsToInvokeLambda:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !Ref "LambdaFunction"
Action: "lambda:InvokeFunction"
Principal: "events.amazonaws.com"
SourceArn:
Fn::GetAtt:
- "ScheduledRule"
- "Arn"
I have a very confusing issue. I am trying to create s3 bucket with an event attached to it trigger lambda. Here is my code:
#s3-test-bucket
---
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
project:
Description: project
Type: String
ConstraintDescription: Any string
EnvironmentApp:
Description: EnvironmentApp
Type: String
ConstraintDescription: Any string
S3BucketName:
Description: EnvironmentApp
Type: String
ConstraintDescription: Any string
Resources:
S3Bucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Sub images-${EnvironmentApp}
NotificationConfiguration:
LambdaConfigurations:
-
Function: arn:aws:lambda:us-east-1:xxxxxxxxxxx:function:test-
trigger-cfn
Event: "s3:ObjectCreated:*"
Filter:
S3Key:
Rules:
-
Name: suffix
Value: zip
DeletionPolicy: Delete
Now the problem is when I run it I get the following error:
10:25:56 UTC-0300 CREATE_FAILED AWS::S3::Bucket S3Bucket Unable to validate the following destination configurations
I have my lambda created before running the stack so what could be the cause of the issue?
Update: this fixed the issue :
LambdaPolicy:
DependsOn:
- Lambda
Type: AWS::Lambda::Permission
Properties:
FunctionName:
"Fn::GetAtt": [ LambdaImageResizer, Arn ]
Action: "lambda:InvokeFunction"
Principal: "s3.amazonaws.com"
SourceArn: arn:aws:s3:::xxxxx
Please check if you have allowed invokeFunction permission on your lambda function.
This looks similar issue as stated in AWS forum
https://forums.aws.amazon.com/thread.jspa?threadID=167470