Here is example for lambda invocation by event:
EventRule:
Type: AWS::Events::Rule
Properties:
Description: "EventRule"
EventPattern:
source:
- "aws.ec2"
detail-type:
- "EC2 Instance State-change Notification"
detail:
state:
- "stopping"
State: "ENABLED"
Targets:
-
Arn:
Fn::GetAtt:
- "LambdaFunction"
- "Arn"
Id: "TargetFunctionV1"
PermissionForEventsToInvokeLambda:
Type: AWS::Lambda::Permission
Properties:
FunctionName:
Ref: "LambdaFunction"
Action: "lambda:InvokeFunction"
Principal: "events.amazonaws.com"
SourceArn:
Fn::GetAtt:
- "EventRule"
- "Arn"
How to modify this tp be able to invoke state machine not lambda? I have replaced "LambdaFunction with "MyStateMachine"
but how modify PermissionForEventsToInvokeLambda? There is no AWS::StepFunctions::Permission in cloudformation :-(
Here's an snippet from the template we used to have to trigger state machine processing from an S3 event:
InboundBucketPutObjectRule:
Type: 'AWS::Events::Rule'
Properties:
Name: !Sub 'inbound-bucket-put-object-rule'
EventPattern:
source:
- aws.s3
detail-type:
- AWS API Call via CloudTrail
detail:
eventSource:
- s3.amazonaws.com
eventName:
- PutObject
- CopyObject
- CompleteMultipartUpload
requestParameters:
bucketName:
- !Ref InboundBucket
Targets:
- Id: ProcessNewObject
Arn: !Ref StateMachine
RoleArn: !GetAtt
- StateMachineStartExecutionRole
- Arn
StateMachineStartExecutionPolicy:
Type: 'AWS::IAM::ManagedPolicy'
Properties:
ManagedPolicyName: !Sub 'state-machine-start-execution'
Roles:
- !Ref StateMachineStartExecutionRole
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- 'states:StartExecution'
Resource:
- !Ref StateMachine
StateMachineStartExecutionRole:
Type: 'AWS::IAM::Role'
Properties:
RoleName: !Sub 'state-machine-start-execution'
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: events.amazonaws.com
Action: 'sts:AssumeRole'
StateMachine:
Type: 'AWS::StepFunctions::StateMachine'
Properties:
StateMachineName: !Sub 'state-machine'
Taken from real template so I had to anonymize this which might have introduced errors.
Template structure:
InboundBucketPutObjectRule is the CloudWatch event which gets triggered on file uploads.
StateMachineStartExecutionPolicy + StateMachineStartExecutionRole essentially allow CloudWatch event to start execution of a state machine.
StateMachine - the state machine which should be started (definition omitted).
I wrote above that we used to have this - but not anymore. Please see the following answer for our current solution:
https://stackoverflow.com/a/57563395/303810
Related
I am pretty new to CloudFormation and playing around with it. I am trying to create a ConfigRule that checks if an EC2 instance does not have monitoring enabled. If it is compliant or non-compliant it will then notify the user via email (SNS). It creates the resources with no errors and even sends a subscription email which I then subscribe to. However, if I change the monitoring of the ec2 to enabled or disabled, no email notification comes to my email. I tried this same scenario manually through the console and had no issues getting email notifications.
CF Designer
Here is my code:
AWSTemplateFormatVersion: "2010-09-09"
Description: ""
Parameters:
TopicName:
Type: String
Description: Topic Name
Default: my-topic
Resources:
snsTopic:
Type: AWS::SNS::Topic
Properties:
DisplayName: SNS EC2 Topic
Subscription:
- Endpoint: "test#gmail.com"
Protocol: email
TopicName: my-topic2
snsSubscription:
Type: AWS::SNS::Subscription
Properties:
TopicArn: !Ref snsTopic
Protocol: email
Endpoint: test#gmail.com
ConfigRule:
Type: "AWS::Config::ConfigRule"
DependsOn: ConfigurationRecorder
Properties:
ConfigRuleName: "ec2-monitoring-check"
Description: "A Config rule that checks if an instance has monitoring enabled"
Scope:
ComplianceResourceTypes:
- "AWS::EC2::Instance"
Source:
Owner: "AWS"
SourceIdentifier: "EC2_INSTANCE_DETAILED_MONITORING_ENABLED"
ConfigurationRecorder:
Type: "AWS::Config::ConfigurationRecorder"
Properties:
RoleARN:
Fn::GetAtt:
- "IamRoleForAwsConfig"
- "Arn"
RecordingGroup:
AllSupported: true
IncludeGlobalResourceTypes: true
DeliveryChannel:
Type: "AWS::Config::DeliveryChannel"
Properties:
S3BucketName:
Ref: "S3BucketForAwsConfig"
SnsTopicARN: !Ref snsTopic
S3BucketForAwsConfig:
Type: "AWS::S3::Bucket"
Properties: {}
IamRoleForAwsConfig:
Type: "AWS::IAM::Role"
Properties:
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/service-role/AWSConfigRole"
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Sid: ""
Effect: "Allow"
Principal:
Service: "config.amazonaws.com"
Action: "sts:AssumeRole"
Policies:
- PolicyName: "allow-access-to-config-s3-bucket"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Action:
- "s3:PutObject"
Resource:
- Fn::Join:
- ""
-
- Fn::GetAtt:
- "S3BucketForAwsConfig"
- "Arn"
- "/*"
Condition:
StringLike:
s3:x-amz-acl: "bucket-owner-full-control"
- Effect: "Allow"
Action:
- "s3:GetBucketAcl"
Resource:
Fn::GetAtt:
- "S3BucketForAwsConfig"
- "Arn"
myEC2Instance:
Type: AWS::EC2::Instance
Properties:
ImageId: ami-0cff7528ff583bf9a
InstanceType: t2.micro
Monitoring: false
UserData: !Base64 |
#!/bin/bash -ex
# put your script here
Tags:
- Key: Name
Value: TestEC2Instance
Metadata: {}
Conditions: {}
Outputs: {}
I tried a similar thing with CloudWatch events and the same thing occurs. I can subscribe to it but when the rule detects something, I never get the subsequent email notifications.
Anyone run into this issue or know how they can help? Thank you.
I have event rule configured in multiple accounts in same region which will get the events from cloudtrail and trigger the events. Since there are so many accounts I cannot configure SNS topic in each and every account and confirm the subscription manually.
How do I have a centralized SNS topic in one account so that the event rule will be sent from all the accounts to that one topic which can trigger the alerts? I haven't worked on this topic earlier and hoping to get some light.
Note: I want to achieve this in CF template.
Here is the cf template that I use.
AWSTemplateFormatVersion: 2010-09-09
Description: ''
Parameters:
DestinationEmailAddress:
Description: ''
Type: String
AllowedPattern: ^[A-Za-z._%+-]+#[A-Za-z.-]+\.[A-za-z]{2,}$
SNSTopicName:
Description: The name of the SNS Topic.
Type: String
AllowedPattern: ^[\w+=,.#/-]+$
Resources:
SnsTopic:
Type: 'AWS::SNS::Topic'
Properties:
Subscription:
- Endpoint: !Ref DestinationEmailAddress
Protocol: email
TopicName: !Ref SNSTopicName
TopicPolicy:
Type: AWS::SNS::TopicPolicy
Properties:
PolicyDocument:
Version: 2012-10-17
Statement:
- Sid: DefaultStatement
Effect: Allow
Principal:
Service:
- events.amazonaws.com
- cloudwatch.amazonaws.com
- config.amazonaws.com
- cloudformation.amazonaws.com
Action:
- "SNS:GetTopicAttributes"
- "SNS:SetTopicAttributes"
- "SNS:AddPermission"
- "SNS:RemovePermission"
- "SNS:DeleteTopic"
- "SNS:Subscribe"
- "SNS:ListSubscriptionsByTopic"
- "SNS:Publish"
- "SNS:Receive"
Resource: !Ref SnsTopic
Condition:
StringEquals:
"AWS:SourceOwner": 'AWS::'
- Sid: AllowEventRuletoPosttoSNSTopic
Effect: Allow
Principal:
Service: "events.amazonaws.com"
Action: "sns:Publish"
Resource: !Ref SnsTopic
Topics:
- !Ref SnsTopic
EventRule:
Type: 'AWS::Events::Rule'
Properties:
Name: ''
Description: ''
State: ENABLED
Targets:
- Arn: !Ref SnsTopic
Id: SNSTopic
EventPattern:
source:
- aws.kms
detail-type:
- AWS API Call via CloudTrail
detail:
eventSource:
- kms.amazonaws.com
eventName:
- DisableKey
Currently my serverless.yml file looks like this:
service: bbb
provider:
name: aws
runtime: go1.x
stage: dev
package:
exclude:
- ./**
include:
- ./bin/**
functions:
ccc:
handler: bin/executable
name: my1minutelambda
role:
'Fn::GetAtt':
- mylambdaexecutionrole
- Arn
resources:
Resources:
mylambdaexecutionrole:
Type: AWS::IAM::Role
Properties:
RoleName: my-basiclambdaexec-role
Description: This is my basiclambdaexecution role
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Principal:
Service:
- "lambda.amazonaws.com"
Action:
- "sts:AssumeRole"
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
myminschedulerevent:
Type: AWS::Events::Rule
Properties:
Description: This is my 1 minute rate scheduler.
Name: my-1-min-trigger-event-scheduler
ScheduleExpression: rate(1 hour)
Targets:
-
Arn: "arn:aws:lambda:us-east-1:111111111111:function:my1minutelambda" #update your a/c Id
Id: "TargetFunctionV1"
command used to deploy: sls deploy
After deployment finished, I can see on aws management console that all my resources got created.
BUT I am not able to see cloudwatch trigger extablishment for my lambda function.
See below screenshot:
CloudWatch Event Rule created successfully. (Target section pointing to my lambda function)
Trigger link not established for my lambda:
Please let me know what i am missing here. Thank you.
Update#1:
After adding following lines (as suggested by Marcin), I am able to see "CloudWatch event".
EventsPermission:
Type: AWS::Lambda::Permission
Properties:
FunctionName: my1minutelambda
Action: lambda:InvokeFunction
Principal: events.amazonaws.com
SourceAccount: !Ref 'AWS::AccountId'
SourceArn: !GetAtt myminschedulerevent.Arn
But, I can't see CloudWatch logs!! So, I can't findout if my lambda function is executing. Please see image below:
I tried to replicate the issue using serverless framework.
To do so I added the following AWS::Lambda::Permission to the end of your template:
EventsPermission:
Type: AWS::Lambda::Permission
Properties:
FunctionName: dsfgsdfg # <-- REPLACE this with your function name my1minutelambda
Action: lambda:InvokeFunction
Principal: events.amazonaws.com
SourceArn: !GetAtt myminschedulrevent.Arn
After adding the permissions, the console showed the trigger as expected:
If all you are trying to do is get a Lambda function to execute on a schedule, the Serverless Framework already includes an event type expressly for that purpose:
functions:
crawl:
handler: crawl
events:
- schedule: rate(2 hours)
- schedule: cron(0 12 * * ? *)
It will set everything up for you with no need to add additional CloudFormation. You can find the documentation here: https://www.serverless.com/framework/docs/providers/aws/events/schedule/#schedule/
ScheduledRule:
Type: AWS::Events::Rule
Properties:
Name: "SphEvent"
Description: "ScheduledRule"
ScheduleExpression: "rate(1 hour)"
State: "ENABLED"
Targets:
- Arn: !Sub "arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:lambda-name"
Id: "TargetFunctionV1"
PermissionForEventsToInvokeLambda:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !Ref "LambdaFunction"
Action: "lambda:InvokeFunction"
Principal: "events.amazonaws.com"
SourceArn:
Fn::GetAtt:
- "ScheduledRule"
- "Arn"
Considering this lambda function on a serverless.yml file:
functions:
s3toEc2Lambda:
handler: s3toec2lambda.S3toEc2Lambda
name: "${self:service}-s3toEc2Lambda"
role: S3toEc2LambdaRole
And considering this SNS created on resources section: Does someone knows how to inform the Sns ARN Endpoint from the lambda function s3toEc2Lambda ?
resources:
Resources:
WordpressFrontEndSnsS3toEc2:
Type: AWS::SNS::Topic
Properties:
TopicName: "wordpress-front-end-s3-ec2"
WordpressFrontEndSnsS3toEc2Lambda:
Type: AWS::SNS::Subscription
Properties:
Endpoint: { "Fn::GetAtt": ["s3toEc2Lambda", "Arn" ] } <------ HERE <------
#Endpoint: ${self:functions.s3toEc2Lambda} <------ OR HERE <------
#Endpoint: { "Fn::GetAtt": ["${self:functions.s3toEc2Lambda}", "Arn" ] } <------ OR HERE <------
Protocol: lambda
TopicArn: !Ref 'WordpressFrontEndSnsS3toEc2'
For me always appear a error message like this: "Template error: instance of Fn::GetAtt references undefined resource s3toEc2Lambda"
Thank You !
CloudFormation resources created by serverless have known format. For lambda function this is:
{normalizedFunctionName}LambdaFunction
Thus you should be able to reference your function using the following:
"Fn::GetAtt": [ S3toEc2LambdaLambdaFunction, Arn ]
More example about this are here
We can create Function Roles, Functions Policy and Lambda functions SAM template.yml file by this
Type: AWS::IAM::Role
Properties:
RoleName: UatAdminUserStatsLambda
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: 'Allow'
Principal:
Service:
- 'lambda.amazonaws.com'
Action:
- 'sts:AssumeRole'
FunctionPolicy:
Type: AWS::IAM::Policy
DependsOn: FunctionRole
Properties:
PolicyName: UserStatsPolicy
Roles:
- !Ref FunctionRole
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: 'Allow'
Action:
- 'iam:GetUser'
- 'logs:CreateLogGroup'
- 'logs:CreateLogStream'
- 'logs:GetLogEvents'
- 'logs:PutLogEvents'
- 's3:GetObject'
- 's3:PutObject'
Resource: '*'
adminUsersList:
Type: AWS::Serverless::Function
Properties:
CodeUri: src/admin/
Role: !GetAtt FunctionRole.Arn
Handler: adminUsersList.adminUsersList
Layers:
- !Ref NodeDependenciesLayer
Events:
adminUsersListEvent:
Type: Api
Properties:
Path: /api/admins
Method: GET
I'm creating an ASG group which has a lifecyclehook for termination:
LifecycleHook:
Type: AWS::AutoScaling::LifecycleHook
Properties:
AutoScalingGroupName: !Ref NodeGroup
DefaultResult: CONTINUE
HeartbeatTimeout: 60
LifecycleHookName: !Sub "${AWS::StackName}-lifecycle-hook"
LifecycleTransition: autoscaling:EC2_INSTANCE_TERMINATING
Now I create a lambda function as well:
LambdaCreation:
Type: "AWS::Lambda::Function"
Properties:
Handler: "lambda_function.lambda_handler"
Environment:
Variables:
aws_region : !Ref AWSRegion
Role: !GetAtt LambdaExecutionRole.Arn
Code:
S3Bucket: !Ref LambdaCodeBucket
S3Key: "lambda-functions/function.zip"
Runtime: "python3.6"
Timeout: 60
On cloudwatch events, i created a rule for said event:
CloudwatchEvent:
Type: AWS::Events::Rule
Properties:
Description: ASG scale-in event to lambda
EventPattern: {
"source": [
"aws.autoscaling"
],
"detail-type": [
"EC2 Instance-terminate Lifecycle Action"
],
"detail": {
"AutoScalingGroupName":
[
{
"Fn::ImportValue" :
{
"Fn::Sub" : "${RootStackName}-nodes-asg-name"
}
}
]
}
}
State: ENABLED
Targets:
-
Arn:
!GetAtt LambdaCreation.Arn
Id:
!Ref LambdaCreation
But the lambda is never triggered.
Now, on AWS console I don't see a trigger on the designer. But if i add manually a cloudwatch trigger for the created rule, it starts working...
Why is the trigger on the lambda side not created? What am I missing?
Thanks all!
I faced the exact same frustration. Only difference is that I was using terraform but that's irrelavant.
You are missing this:
{
"Type" : "AWS::Lambda::Permission",
"Properties" : {
"Action" : String,
"EventSourceToken" : String,
"FunctionName" : String,
"Principal" : String,
"SourceAccount" : String,
"SourceArn" : String
}
}
The reason the "manual way" works because it creates the trigger AND the permission. When you provision stuff using IaC tools like Cloudformation/terraform, you need to explicitly specify this Lambda permission object.
The below code snippet creates a lambda function and creates a cloudwatch event to trigger the lambda function with necessary privileges.
LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: root
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:*
Resource: arn:aws:logs:*:*:*
- Effect: Allow
Action:
- s3:ListBucket
Resource: !Join [ '', [ 'arn:aws:s3:::', !Ref LambdaS3Bucket ] ]
- Effect: Allow
Action:
- s3:GetObject
Resource: !Join [ '', [ 'arn:aws:s3:::', !Ref LambdaS3Bucket, '/*' ] ]
- Effect: Allow
Action:
- sts:GetCallerIdentity
Resource: '*'
LambdaFunction:
Type: "AWS::Lambda::Function"
Properties:
Description: "Lambda function"
FunctionName: !Ref LambdaFunctionName
Handler: !Ref LambdaHandler
Runtime: !Ref LambdaRuntime
Timeout: !Ref LambdaTimeout
MemorySize: !Ref LambdaMemorysize
Role: !GetAtt LambdaExecutionRole.Arn
Code:
S3Bucket: !Ref LambdaS3Bucket
S3Key: !Ref LambdaS3BucketKey
Environment:
Variables:
time_interval_in_hours: !Ref TimeIntervalInHours
DependsOn: LambdaExecutionRole
CleanupEventRule:
Type: AWS::Events::Rule
Properties:
Description: "Cloudwatch Rule"
ScheduleExpression: !Ref CloudwatchScheduleExpression
State: !Ref CloudWatchEventState
Targets:
- Arn: !Sub ${LambdaFunction.Arn}
Id: "CleanupEventRule"
DependsOn: LambdaFunction
LambdaSchedulePermission:
Type: AWS::Lambda::Permission
Properties:
Action: 'lambda:InvokeFunction'
FunctionName: !Sub ${LambdaFunction.Arn}
Principal: 'events.amazonaws.com'
SourceArn: !Sub ${CleanupEventRule.Arn}
DependsOn: LambdaFunction