I know I can create a Scheduled Cloud Watch event by means of AWS Console:
https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/Create-CloudWatch-Events-Scheduled-Rule.html
Is there a way to declare the similar event in Cloud Formation template?
Below is the example to create a scheduled event in cloudwatch, It creates a rule that invokes the specified Lambda function every 10 minutes. The PermissionForEventsToInvokeLambda resource grants EventBridge permission to invoke the associated function.
"ScheduledRule": {
"Type": "AWS::Events::Rule",
"Properties": {
"Description": "ScheduledRule",
"ScheduleExpression": "rate(10 minutes)",
"State": "ENABLED",
"Targets": [{
"Arn": { "Fn::GetAtt": ["LambdaFunction", "Arn"] },
"Id": "TargetFunctionV1"
}]
}
},
"PermissionForEventsToInvokeLambda": {
"Type": "AWS::Lambda::Permission",
"Properties": {
"FunctionName": { "Ref": "LambdaFunction" },
"Action": "lambda:InvokeFunction",
"Principal": "events.amazonaws.com",
"SourceArn": { "Fn::GetAtt": ["ScheduledRule", "Arn"] }
}
}
The example is referenced from AWS official documentation.
Yes, it's possible.
The AWS::Events::Rule resource creates a rule that matches incoming Amazon CloudWatch Events (CloudWatch Events) events and routes them to one or more targets for processing.
Here's the sample CloudFormation Snippet:
Type: AWS::Events::Rule
Properties:
Description: String
EventPattern: JSON object
Name: String
ScheduleExpression: String
State: String
Targets:
- Target
Here's the official documentation, if you have more questions.
Yes, It's possible as share by #bhalothia. Please find an article which will give you a deep dive into it.
Practical Implementation:
http://marcelog.github.io/articles/serverless_cloudwatch_event_cloudformation_template.html
Detail dodcumentation:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-events-rule-target.html
I hope this helps you.
Related
I was trying to create an AWS state machine(step functions) using AWS SAM which is triggered by S3 event. Following is my AWS SAM yml snippet.
SampleStateMachine:
Type: AWS::Serverless::StateMachine
Properties:
Name: sample-state-machine
DefinitionUri: state-machines/my-definition.asl.json
Events:
S3PutEvent:
Type: EventBridgeRule
Properties:
Pattern:
source:
- "aws.s3"
detail:
eventSource:
- s3.amazonaws.com
eventName:
- PutObject
requestParameters:
bucketName:
- !Ref MyBucketName
On deploying this application, it's successfully creating the rule with the pattern that I've specified in the sam yml template. (but with a slight change in the order of JSON key-value pairs)
{
"source": [
"aws.s3"
],
"detail": {
"eventSource": [
"s3.amazonaws.com"
],
"requestParameters": {
"bucketName": [
"my-bucket"
]
},
"eventName": [
"PutObject"
]
}
}
Unfortunately, this rule is not capturing any event from the event bus. so I've tried like changing the JSON Key-Value pair in the following order,
{
"source": [ "aws.s3"
],
"detail": {
"eventSource": [
"s3.amazonaws.com"
],
"eventName": [
"PutObject"
],
"requestParameters": {
"bucketName": [
"my-bucket"
]
}
}
}
and it started receiving events and working fine.
So my question is,
Is this order really matters for AWS eventbridge rule pattern?
If so, how we can preserve this order while AWS sam execution(YML to JSON)?
Thanks
Order should not matter. If you can reproduce the issue, you should file a bug report with AWS support to get the service to fix it.
I want to use CloudFormation to create a stack of preexisting Lambda Functions into a State Machine using Step Functions on a schedule (30 mins). I have successfully created the stack for my other methods.
In essence, I need help or guidance on how to create a scheduled event in CloudFormation for Step Functions. Here is what I have been trying:
"NOTDScheduler": {
"Type": "AWS::Events::Rule",
"Properties": {
"Description": "Schedules a NOTD every 30 minutes",
"ScheduleExpression": "rate(30 minutes)",
"State": "ENABLED",
"Targets": [
{
"Arn": "${statemachineARN}",
"statemachineARN": {
"Fn::GetAtt": [
"NOTDStateMachine",
"Arn"
]
},
"Id": "NOTDScheduleTarget"
}
]
},
But I keep getting errors such as
[Error] /Resources/NOTDScheduler/Properties/Targets/0/statemachineARN/Fn::GetAtt: Resource type AWS::StepFunctions::StateMachine does not support attribute {Arn}.
and have no clue how Arn isnt a supported attribute. Is there a workaround?
To get the ARN of a AWS::StepFunctions::StateMachine resource you need to call !Ref NOTDStateMachine instead of !GetAtt NOTDStateMachine.Arn
Check Return Values here: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-stepfunctions-statemachine.html
Having trouble configuring AWS Lambda to be triggered by a Rule->Trigger as a Scheduled Event Source using CloudFormation (in reality, using Python's Troposphere.) This has cost me a couple of days already, and any help would be appreciated.
Here's the relevant CF JSON snippet -
"DataloaderRetrier": {
"Properties": {
"Code": {
"S3Bucket": "mycompanylabs-config",
"S3Key": "v3/mycompany-component-loader-lambda-0.5.jar"
},
"FunctionName": "DataloaderRetriervitest27",
"Handler": "mycompany.ScheduledEventHandler::handleRequest",
"MemorySize": 320,
"Role": "arn:aws:iam::166662328783:role/kinesis-lambda-role",
"Runtime": "java8",
"VpcConfig": {
"SecurityGroupIds": [
"sg-2f1f6047"
],
"SubnetIds": [
"subnet-ec3c1435"
]
}
},
"Type": "AWS::Lambda::Function"
},
"DataloaderRetrierEventTriggerPermission": {
"Properties": {
"Action": "lambda:InvokeFunction",
"FunctionName": {
"Fn::GetAtt": [
"DataloaderRetrier",
"Arn"
]
},
"Principal": "events.amazonaws.com",
"SourceAccount": {
"Ref": "AWS::AccountId"
},
"SourceArn": {
"Fn::GetAtt": [
"DataloaderRetrierEventTriggerRule",
"Arn"
]
}
},
"Type": "AWS::Lambda::Permission"
},
"DataloaderRetrierEventTriggerRule": {
"DependsOn": "DataloaderRetrier",
"Properties": {
"Description": "Reminding the lambda to read from the retry SQS",
"Name": "DataloaderRetrierEventTriggerRulevitest27",
"ScheduleExpression": "rate(1 minute)",
"State": "ENABLED",
"Targets": [
{
"Arn": {
"Fn::GetAtt": [
"DataloaderRetrier",
"Arn"
]
},
"Id": "DataloaderRetrierEventTriggerTargetvitest27",
"Input": "{\"Hey\":\"WAKE UP!\"}"
}
]
},
"Type": "AWS::Events::Rule"
}
The AWS Lambda function shows zero invocations, and the Events->Rules metric shows the correct number of invocations, however they all fail. The Lambda shows the trigger in the Triggers section, and the Rule shows the lambda in its trigger sections. They link up fine.
However, if I go in and manually create the same trigger under the rule in the web console, it will happily start sending events to the Lambda.
PS - here's the troposphere code:
# DATALOADER RETRIER LAMBDA
dataloader_retrier = t.add_resource(awslambda.Function(
"DataloaderRetrier",
Code=awslambda.Code(
"DataloaderRetrierCode",
S3Bucket='mycompanylabs-config',
S3Key='v3/mycompany-snowplow-loader-lambda-0.5.jar'
),
FunctionName=suffix("DataloaderRetrier"),
Handler="mycompany.ScheduledEventHandler::handleRequest",
MemorySize="320",
Role="arn:aws:iam::166662328783:role/kinesis-lambda-role",
Runtime="java8",
VpcConfig=lambda_vpc_config
))
dataloader_retrier_scheduled_rule = t.add_resource(events.Rule(
"DataloaderRetrierEventTriggerRule",
Name=suffix("DataloaderRetrierEventTriggerRule"),
Description="Reminding the lambda to read from the retry SQS",
Targets=[events.Target(
Id=suffix("DataloaderRetrierEventTriggerTarget"),
Arn=tr.GetAtt("DataloaderRetrier", "Arn"),
Input='{"Hey":"WAKE UP!"}'
)],
State='ENABLED',
ScheduleExpression="rate(1 minute)",
DependsOn="DataloaderRetrier"
)),
t.add_resource(awslambda.Permission(
"DataloaderRetrierEventTriggerPermission",
Action="lambda:InvokeFunction",
FunctionName=tr.GetAtt("DataloaderRetrier", "Arn"),
Principal="events.amazonaws.com",
SourceAccount=tr.Ref("AWS::AccountId"),
SourceArn=tr.GetAtt("DataloaderRetrierEventTriggerRule", "Arn")
))
You need to remove the SourceAccount parameter from your AWS::Lambda::Permission Resource.
As described in the AddPermission API documentation, the SourceAccount parameter restricts the 'source' of the permitted invocation to the specified AWS Account ID, for example when specifying an S3 Bucket or CloudWatch Logs notification.
However (and the docs should probably be made more clear on this point), in the case of a CloudWatch Events Schedule Expression, the source of the Event is aws.events, not your own AWS Account ID, which is why adding this parameter causes the event to fail to trigger the Lambda function.
I've got an AWS CloudFormation template that creates an SNS topic and a subscription:
"AcceptedTopic":{
"Type": "AWS::SNS::Topic",
"Properties": {
"DisplayName": {"Fn::Join": ["", ["Accepted-", {"Ref": "Env"}]]},
"TopicName": {"Fn::Join": ["", ["Accepted-", {"Ref": "Env"}]]},
"Subscription": [{
"Endpoint": {"Fn::GetAtt" : [ "SomeQueue" , "Arn" ]},
"Protocol": "sqs"
}]
}
}
I need to specify the 'Raw Message Delivery' subscription attribute. How can I do that in AWS CloudFormation?
As of this writing, AWS CloudFormation doesn't support that natively. As an alternate, you can create a Lambda-backed custom resource to get around this limitation and set that attribute using set-subscription-attributes instead. Here are some helpful resources to help accomplish that:
Lambda-backed custom resources
SNS' set-subscription-attributes API
Now AWS CloudFormation supports it with AWS::SNS::Subscription. So instead of adding the subscription as a property of the topic, add an Subscription resource linked above.
A caveat though, is that if you already created a topic with that subscription and are now trying to add the attribute, it'd fail miserably with Invalid Parameter error. The cause is it's considering the standalone Subscription added in the template as a new resource and trying to create it. I haven't found a good way around this other than deleting that subscription manually, which is not good practice in production environment.
My solution around this is separating it into 2 steps. First, remove the property subscription from the topic and add a Subscription resource. Then, add new attributes to the subscription resource.
First:
{
"AcceptedTopic": {
"Type": "AWS::SNS::Topic",
"Properties": {
"DisplayName": {
"Fn::Join": ["", ["Accepted-", {"Ref": "Env"}]]
},
"TopicName": {
"Fn::Join": ["", ["Accepted-", {"Ref": "Env"}]]
}
}
}
"AcceptedTopicSubscription": {
"TopicArn": { "Ref": "AcceptedTopic" },
"Endpoint": {
"Fn::GetAtt": ["SomeQueue", "Arn"]
},
"Protocol": "Sqs"
}
}
Then:
{
...
"AcceptedTopicSubscription": {
"TopicArn": { "Ref": "AcceptedTopic" },
"Endpoint": {
"Fn::GetAtt": ["SomeQueue", "Arn"]
},
"Protocol": "Sqs",
"RawMessageDelivery": "true"
}
}
I'm trying to create a Lambda notification via CloudFormation but getting an error about the ARN format being incorrect.
Either my CloudFormation is wrong or it doesn't support the Lambda preview yet.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Parameters": {
"LambdaArn": {
"Type": "String",
"Default": "arn:aws:lambda:{some-region}:{some-account-id}:function:{some-fn-name}"
}
},
"Resources": {
"EventArchive": {
"Type": "AWS::S3::Bucket",
"Properties": {
"NotificationConfiguration": {
"TopicConfigurations": [
{
"Event": "s3:ObjectCreated:Put",
"Topic": {
"Ref": "LambdaArn"
}
}
]
}
}
}
}
}
But when I push up this CloudFormation I get the message:
The ARN is not well formed
Does anyone have idea as to what this means? I know the example above has been modified so not to use my actual ARN, but in my actual code I've copied the ARN directly from the GUI.
Also, interestingly I was able to create the notification via the AWS console, and so I just assume that AWS CloudFormation doesn't yet support this feature (even though that's not quite clear I don't think when reading the documentation).
It looks like AWS has now released support for notifying lambda functions directly in CloudFormation.
The S3 NotificationConfiguration definition used to only include TopicConfigurations but has been updated to include LambdaConfigurations as well.
After adding the NoficationConfiguration, make sure you include a Lambda::Permission resource so that S3 is allowed to execute your lambda function. Here is an example permission that can be used as a template:
"PhotoBucketExecuteProcessorPermission": {
"Type" : "AWS::Lambda::Permission",
"Properties" : {
"Action":"lambda:invokeFunction",
"FunctionName": { "Fn::GetAtt": [ "PhotoProcessor", "Arn" ]},
"Principal": "s3.amazonaws.com",
"SourceAccount": {"Ref" : "AWS::AccountId" },
"SourceArn": {
"Fn::Join": [":", [
"arn","aws","s3","", ""
,{"Ref" : "PhotoBucketName"}]]
}
}
}
From the docs:
The Amazon SNS topic to which Amazon S3 reports the specified events.
It appears that although S3 supports sending events to Lambda, CloudFormation has not yet caught up. It expects an SNS ARN where you are providing a Lambda function ARN.
For now, it looks like you will have to hook up the event notification manually.