I'm creating an ASG group which has a lifecyclehook for termination:
LifecycleHook:
Type: AWS::AutoScaling::LifecycleHook
Properties:
AutoScalingGroupName: !Ref NodeGroup
DefaultResult: CONTINUE
HeartbeatTimeout: 60
LifecycleHookName: !Sub "${AWS::StackName}-lifecycle-hook"
LifecycleTransition: autoscaling:EC2_INSTANCE_TERMINATING
Now I create a lambda function as well:
LambdaCreation:
Type: "AWS::Lambda::Function"
Properties:
Handler: "lambda_function.lambda_handler"
Environment:
Variables:
aws_region : !Ref AWSRegion
Role: !GetAtt LambdaExecutionRole.Arn
Code:
S3Bucket: !Ref LambdaCodeBucket
S3Key: "lambda-functions/function.zip"
Runtime: "python3.6"
Timeout: 60
On cloudwatch events, i created a rule for said event:
CloudwatchEvent:
Type: AWS::Events::Rule
Properties:
Description: ASG scale-in event to lambda
EventPattern: {
"source": [
"aws.autoscaling"
],
"detail-type": [
"EC2 Instance-terminate Lifecycle Action"
],
"detail": {
"AutoScalingGroupName":
[
{
"Fn::ImportValue" :
{
"Fn::Sub" : "${RootStackName}-nodes-asg-name"
}
}
]
}
}
State: ENABLED
Targets:
-
Arn:
!GetAtt LambdaCreation.Arn
Id:
!Ref LambdaCreation
But the lambda is never triggered.
Now, on AWS console I don't see a trigger on the designer. But if i add manually a cloudwatch trigger for the created rule, it starts working...
Why is the trigger on the lambda side not created? What am I missing?
Thanks all!
I faced the exact same frustration. Only difference is that I was using terraform but that's irrelavant.
You are missing this:
{
"Type" : "AWS::Lambda::Permission",
"Properties" : {
"Action" : String,
"EventSourceToken" : String,
"FunctionName" : String,
"Principal" : String,
"SourceAccount" : String,
"SourceArn" : String
}
}
The reason the "manual way" works because it creates the trigger AND the permission. When you provision stuff using IaC tools like Cloudformation/terraform, you need to explicitly specify this Lambda permission object.
The below code snippet creates a lambda function and creates a cloudwatch event to trigger the lambda function with necessary privileges.
LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: root
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:*
Resource: arn:aws:logs:*:*:*
- Effect: Allow
Action:
- s3:ListBucket
Resource: !Join [ '', [ 'arn:aws:s3:::', !Ref LambdaS3Bucket ] ]
- Effect: Allow
Action:
- s3:GetObject
Resource: !Join [ '', [ 'arn:aws:s3:::', !Ref LambdaS3Bucket, '/*' ] ]
- Effect: Allow
Action:
- sts:GetCallerIdentity
Resource: '*'
LambdaFunction:
Type: "AWS::Lambda::Function"
Properties:
Description: "Lambda function"
FunctionName: !Ref LambdaFunctionName
Handler: !Ref LambdaHandler
Runtime: !Ref LambdaRuntime
Timeout: !Ref LambdaTimeout
MemorySize: !Ref LambdaMemorysize
Role: !GetAtt LambdaExecutionRole.Arn
Code:
S3Bucket: !Ref LambdaS3Bucket
S3Key: !Ref LambdaS3BucketKey
Environment:
Variables:
time_interval_in_hours: !Ref TimeIntervalInHours
DependsOn: LambdaExecutionRole
CleanupEventRule:
Type: AWS::Events::Rule
Properties:
Description: "Cloudwatch Rule"
ScheduleExpression: !Ref CloudwatchScheduleExpression
State: !Ref CloudWatchEventState
Targets:
- Arn: !Sub ${LambdaFunction.Arn}
Id: "CleanupEventRule"
DependsOn: LambdaFunction
LambdaSchedulePermission:
Type: AWS::Lambda::Permission
Properties:
Action: 'lambda:InvokeFunction'
FunctionName: !Sub ${LambdaFunction.Arn}
Principal: 'events.amazonaws.com'
SourceArn: !Sub ${CleanupEventRule.Arn}
DependsOn: LambdaFunction
Related
I am pretty new to CloudFormation and playing around with it. I am trying to create a ConfigRule that checks if an EC2 instance does not have monitoring enabled. If it is compliant or non-compliant it will then notify the user via email (SNS). It creates the resources with no errors and even sends a subscription email which I then subscribe to. However, if I change the monitoring of the ec2 to enabled or disabled, no email notification comes to my email. I tried this same scenario manually through the console and had no issues getting email notifications.
CF Designer
Here is my code:
AWSTemplateFormatVersion: "2010-09-09"
Description: ""
Parameters:
TopicName:
Type: String
Description: Topic Name
Default: my-topic
Resources:
snsTopic:
Type: AWS::SNS::Topic
Properties:
DisplayName: SNS EC2 Topic
Subscription:
- Endpoint: "test#gmail.com"
Protocol: email
TopicName: my-topic2
snsSubscription:
Type: AWS::SNS::Subscription
Properties:
TopicArn: !Ref snsTopic
Protocol: email
Endpoint: test#gmail.com
ConfigRule:
Type: "AWS::Config::ConfigRule"
DependsOn: ConfigurationRecorder
Properties:
ConfigRuleName: "ec2-monitoring-check"
Description: "A Config rule that checks if an instance has monitoring enabled"
Scope:
ComplianceResourceTypes:
- "AWS::EC2::Instance"
Source:
Owner: "AWS"
SourceIdentifier: "EC2_INSTANCE_DETAILED_MONITORING_ENABLED"
ConfigurationRecorder:
Type: "AWS::Config::ConfigurationRecorder"
Properties:
RoleARN:
Fn::GetAtt:
- "IamRoleForAwsConfig"
- "Arn"
RecordingGroup:
AllSupported: true
IncludeGlobalResourceTypes: true
DeliveryChannel:
Type: "AWS::Config::DeliveryChannel"
Properties:
S3BucketName:
Ref: "S3BucketForAwsConfig"
SnsTopicARN: !Ref snsTopic
S3BucketForAwsConfig:
Type: "AWS::S3::Bucket"
Properties: {}
IamRoleForAwsConfig:
Type: "AWS::IAM::Role"
Properties:
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/service-role/AWSConfigRole"
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Sid: ""
Effect: "Allow"
Principal:
Service: "config.amazonaws.com"
Action: "sts:AssumeRole"
Policies:
- PolicyName: "allow-access-to-config-s3-bucket"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Action:
- "s3:PutObject"
Resource:
- Fn::Join:
- ""
-
- Fn::GetAtt:
- "S3BucketForAwsConfig"
- "Arn"
- "/*"
Condition:
StringLike:
s3:x-amz-acl: "bucket-owner-full-control"
- Effect: "Allow"
Action:
- "s3:GetBucketAcl"
Resource:
Fn::GetAtt:
- "S3BucketForAwsConfig"
- "Arn"
myEC2Instance:
Type: AWS::EC2::Instance
Properties:
ImageId: ami-0cff7528ff583bf9a
InstanceType: t2.micro
Monitoring: false
UserData: !Base64 |
#!/bin/bash -ex
# put your script here
Tags:
- Key: Name
Value: TestEC2Instance
Metadata: {}
Conditions: {}
Outputs: {}
I tried a similar thing with CloudWatch events and the same thing occurs. I can subscribe to it but when the rule detects something, I never get the subsequent email notifications.
Anyone run into this issue or know how they can help? Thank you.
I wanted to use the ARN as parameter input to cloudformation stack resources EventRuleRegion1 - Target as well as EventBridgeIAMrole , but it is not working. when i call with Ref function
Original ARN
arn:aws:events:ap-southeast-2:123456789123:event-bus/central-eventbus-sydney
When i give the arn directly in code its working fine.
Code
AWSTemplateFormatVersion: 2010-09-09
Parameters:
EventBridgeName:
Description: Enter the Event Bridge Name
Type: String
Default: ec2-lifecycle-events
EventBusName:
Description: Enter the Central Event Bus Name
Type: String
Default: central-eventbus-sydney
EventBusArn:
Description: Enter the ARN of Central Event Bus
Type: String
Default: arn:aws:events:ap-southeast-2:123456789123:event-bus/central-eventbus-sydney
Monitoringaccount:
Description: Enter the Monitoring AWS account number
Type: String
Default: 123456789123
Resources:
EventRuleRegion1:
Type: AWS::Events::Rule
Properties:
Description: Event rule to send events to monitoring account event bus
EventBusName: default
EventPattern:
source:
- aws.ec2
detail-type:
- "EC2 Instance State-change Notification"
detail:
state:
- "running"
- "stopped"
- "terminated"
Name: !Ref EventBridgeName
State: ENABLED
Targets:
- Arn: >-
- !Join [ "", [ !Sub "arn:aws:events:${AWS::Region}:123456789123:event-bus/",!Ref EventBusName ] ]
Id: !Ref EventBusName
RoleArn: !GetAtt
- EventBridgeIAMrole
- Arn
EventBridgeIAMrole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: !Sub events.amazonaws.com
Action: 'sts:AssumeRole'
Path: /
Policies:
- PolicyName: PutEventsDestinationBus
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- 'events:PutEvents'
Resource:
- >-
- !Join [ "", [ !Sub "arn:aws:events:${AWS::Region}:123456789123:event-bus/",!Ref EventBusName ] ]
Error
Parameter - !Join [ "", [ !Sub "arn:aws:events:${AWS::Region}:123456789123:event-bus/",!Ref EventBusName ] ] is not valid. Reason: Provided Arn is not in correct format. (Service: AmazonCloudWatchEvents; Status Code: 400; Error Code: ValidationException; Request ID: 0d52a1d6-095e-44f7-9455-b7481dc4fb8d; Proxy: null)
The use of >- will result in literal strings, not evaluation of your CFN functions (join, ref). It should be:
Targets:
- Arn: !Join [ "", [ !Sub "arn:aws:events:${AWS::Region}:123456789123:event-bus/",!Ref EventBusName ] ]
Considering this lambda function on a serverless.yml file:
functions:
s3toEc2Lambda:
handler: s3toec2lambda.S3toEc2Lambda
name: "${self:service}-s3toEc2Lambda"
role: S3toEc2LambdaRole
And considering this SNS created on resources section: Does someone knows how to inform the Sns ARN Endpoint from the lambda function s3toEc2Lambda ?
resources:
Resources:
WordpressFrontEndSnsS3toEc2:
Type: AWS::SNS::Topic
Properties:
TopicName: "wordpress-front-end-s3-ec2"
WordpressFrontEndSnsS3toEc2Lambda:
Type: AWS::SNS::Subscription
Properties:
Endpoint: { "Fn::GetAtt": ["s3toEc2Lambda", "Arn" ] } <------ HERE <------
#Endpoint: ${self:functions.s3toEc2Lambda} <------ OR HERE <------
#Endpoint: { "Fn::GetAtt": ["${self:functions.s3toEc2Lambda}", "Arn" ] } <------ OR HERE <------
Protocol: lambda
TopicArn: !Ref 'WordpressFrontEndSnsS3toEc2'
For me always appear a error message like this: "Template error: instance of Fn::GetAtt references undefined resource s3toEc2Lambda"
Thank You !
CloudFormation resources created by serverless have known format. For lambda function this is:
{normalizedFunctionName}LambdaFunction
Thus you should be able to reference your function using the following:
"Fn::GetAtt": [ S3toEc2LambdaLambdaFunction, Arn ]
More example about this are here
We can create Function Roles, Functions Policy and Lambda functions SAM template.yml file by this
Type: AWS::IAM::Role
Properties:
RoleName: UatAdminUserStatsLambda
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: 'Allow'
Principal:
Service:
- 'lambda.amazonaws.com'
Action:
- 'sts:AssumeRole'
FunctionPolicy:
Type: AWS::IAM::Policy
DependsOn: FunctionRole
Properties:
PolicyName: UserStatsPolicy
Roles:
- !Ref FunctionRole
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: 'Allow'
Action:
- 'iam:GetUser'
- 'logs:CreateLogGroup'
- 'logs:CreateLogStream'
- 'logs:GetLogEvents'
- 'logs:PutLogEvents'
- 's3:GetObject'
- 's3:PutObject'
Resource: '*'
adminUsersList:
Type: AWS::Serverless::Function
Properties:
CodeUri: src/admin/
Role: !GetAtt FunctionRole.Arn
Handler: adminUsersList.adminUsersList
Layers:
- !Ref NodeDependenciesLayer
Events:
adminUsersListEvent:
Type: Api
Properties:
Path: /api/admins
Method: GET
Here is example for lambda invocation by event:
EventRule:
Type: AWS::Events::Rule
Properties:
Description: "EventRule"
EventPattern:
source:
- "aws.ec2"
detail-type:
- "EC2 Instance State-change Notification"
detail:
state:
- "stopping"
State: "ENABLED"
Targets:
-
Arn:
Fn::GetAtt:
- "LambdaFunction"
- "Arn"
Id: "TargetFunctionV1"
PermissionForEventsToInvokeLambda:
Type: AWS::Lambda::Permission
Properties:
FunctionName:
Ref: "LambdaFunction"
Action: "lambda:InvokeFunction"
Principal: "events.amazonaws.com"
SourceArn:
Fn::GetAtt:
- "EventRule"
- "Arn"
How to modify this tp be able to invoke state machine not lambda? I have replaced "LambdaFunction with "MyStateMachine"
but how modify PermissionForEventsToInvokeLambda? There is no AWS::StepFunctions::Permission in cloudformation :-(
Here's an snippet from the template we used to have to trigger state machine processing from an S3 event:
InboundBucketPutObjectRule:
Type: 'AWS::Events::Rule'
Properties:
Name: !Sub 'inbound-bucket-put-object-rule'
EventPattern:
source:
- aws.s3
detail-type:
- AWS API Call via CloudTrail
detail:
eventSource:
- s3.amazonaws.com
eventName:
- PutObject
- CopyObject
- CompleteMultipartUpload
requestParameters:
bucketName:
- !Ref InboundBucket
Targets:
- Id: ProcessNewObject
Arn: !Ref StateMachine
RoleArn: !GetAtt
- StateMachineStartExecutionRole
- Arn
StateMachineStartExecutionPolicy:
Type: 'AWS::IAM::ManagedPolicy'
Properties:
ManagedPolicyName: !Sub 'state-machine-start-execution'
Roles:
- !Ref StateMachineStartExecutionRole
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- 'states:StartExecution'
Resource:
- !Ref StateMachine
StateMachineStartExecutionRole:
Type: 'AWS::IAM::Role'
Properties:
RoleName: !Sub 'state-machine-start-execution'
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: events.amazonaws.com
Action: 'sts:AssumeRole'
StateMachine:
Type: 'AWS::StepFunctions::StateMachine'
Properties:
StateMachineName: !Sub 'state-machine'
Taken from real template so I had to anonymize this which might have introduced errors.
Template structure:
InboundBucketPutObjectRule is the CloudWatch event which gets triggered on file uploads.
StateMachineStartExecutionPolicy + StateMachineStartExecutionRole essentially allow CloudWatch event to start execution of a state machine.
StateMachine - the state machine which should be started (definition omitted).
I wrote above that we used to have this - but not anymore. Please see the following answer for our current solution:
https://stackoverflow.com/a/57563395/303810
I want to create an alert if something goes wrong with Lambda function especially when lambda throws an exception. I am planning to configure SNS topic to send a message if that alert is triggered.
All lambdas are created using CloudFormation scripts, so I am searching for a CloudFormation template to configure alarms on CloudWatch logs. I was not able to find a good/working sample. Sample code below .
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "AWS CloudTrail API Activity Alarm Template for CloudWatch Logs",
"Parameters" : {
"LogGroupName" : {
"Type" : "String",
"Default" : "CloudTrail/DefaultLogGroup",
"Description" : "Enter CloudWatch Logs log group name. Default is CloudTrail/DefaultLogGroup"
},
"Email" : {
"Type" : "String",
"Description" : "Email address to notify when an API activity has triggered an alarm"
}
},
"Resources" : {
"SecurityGroupChangesAlarm": {
"Type": "AWS::CloudWatch::Alarm",
"Properties": {
"AlarmName" : "CloudTrailSecurityGroupChanges",
"AlarmDescription" : "Alarms when an API call is made to create, update or delete a Security Group.",
"AlarmActions" : [{ "Ref" : "AlarmNotificationTopic" }],
"MetricName" : "SecurityGroupEventCount",
"Namespace" : "CloudTrailMetrics",
"ComparisonOperator" : "GreaterThanOrEqualToThreshold",
"EvaluationPeriods" : "1",
"Period" : "300",
"Statistic" : "Sum",
"Threshold" : "1"
}
},
"AlarmNotificationTopic": {
"Type": "AWS::SNS::Topic",
"Properties": {
"Subscription": [
{
"Endpoint": { "Ref": "Email" },
"Protocol": "email"
}
]
}
}
}
}
In order to do this, we need to create a subscription filter on the log group for that lambda with FilterPattern: "Exception"
So whenever there is an Exception word in log message it will trigger a monitor lambda.
Following is a cloudformation template in YAML that I have written
Resources:
LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: 'AllowLambdaAccess'
PolicyDocument:
Version: '2012-10-17'
Statement:
- Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Effect: "Allow"
Resource:
Fn::Join:
- ''
- - 'arn:aws:logs:'
- Ref: AWS::Region
- ':'
- Ref: AWS::AccountId
- ':log-group:/aws/lambda/*'
- Action:
- ec2:DescribeNetworkInterfaces
- ec2:CreateNetworkInterface
- ec2:DeleteNetworkInterface
Effect: "Allow"
Resource: "*"
RoleName: !Sub "${AWS::StackName}-LambdaExecutionRole"
SubscriptionFilter:
Type: "AWS::Logs::SubscriptionFilter"
DependsOn: "LambdaInvokePermission"
Properties:
LogGroupName: !Sub "/aws/lambda/${LogGroupName}"
FilterPattern: "Exception"
DestinationArn:
Fn::GetAtt:
- "LambdaFunction"
- "Arn"
LambdaFunction:
Type: 'AWS::Lambda::Function'
Properties:
Code:
S3Bucket: !Ref S3BucketName
S3Key: !Ref ZipFile
Description: Monitor Lambda Function
Handler: 'index.handler'
MemorySize: 1536
Role: !GetAtt
- LambdaExecutionRole
- Arn
Runtime: nodejs6.10
Environment:
Variables:
SMTP_SERVER: !Ref SMTPServer
SMTP_PORT: !Ref SMTPPort
EMAIL_FROM: !Ref FromEmail
EMAIL_TO: !Ref ToEmail
Timeout: 300
FunctionName: !Sub "${AWS::StackName}-LambdaFunction"
VpcConfig:
SecurityGroupIds: !Split [ ",", !Ref SecurityGroupId ]
SubnetIds: !Split [ ",", !Ref SubnetIds ]
DependsOn:
- LambdaExecutionRole
LambdaInvokePermission:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !Ref "LambdaFunction"
Action: "lambda:InvokeFunction"
Principal: !Sub "logs.${AWS::Region}.amazonaws.com"
SourceArn:
Fn::Join:
- ''
- - 'arn:aws:logs:'
- Ref: AWS::Region
- ':'
- Ref: AWS::AccountId
- !Sub ':log-group:/aws/lambda/${LogGroupName}*'