I've got a serverless file which creates an eventbridge rule on the default event bus:
StepFunctionErrorEvent:
Type: AWS::Events::Rule
Properties:
Name: ${self:custom.resourcePrefix}-step-function-error-event-rule
Description: Event bus rule coordinating what targets receive Step Function error events
EventPattern:
source:
- "aws.states"
"detail-type":
- "Step Functions Execution Status Change"
detail:
state:
- "FAILED"
- "TIMED_OUT"
- "ABORTED"
Targets:
- Arn: ${cf:${self:custom.resourcePrefix}-service-internal-slack-integration.PostSlackMessageLambdaArn}
Id: "ErrorSlackMessage"
DeadLetterConfig:
Arn: !GetAtt DefaultErrorTargetDLQ.Arn
DefaultErrorTargetDLQ:
Type: AWS::SQS::Queue
Properties:
QueueName: ${self:custom.resourcePrefix}-DefaultErrorTargetDL
And in a seperate serverless file which also gets deployed I'm adding the following Lambda permission to pl-us-east-2-pilot-post-slack-message:
resources:
Resources:
TriggerPostSlackMessageLambda:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !GetAtt PostSlackMessageLambdaFunction.Arn
Action: lambda:InvokeFunction
Principal: events.amazonaws.com
SourceArn: !Sub arn:aws:events:${AWS::Region}:${AWS::AccountId}:rule/pl-us-east-2-pilot-step-function-error-event-rule
However despite pl-us-east-2-pilot-post-slack-message lambda having the above listed as a permission under 'Resource based policy' (in the Lambda console) the EventBridge rule does not trigger when there is a Lambda failure. It does trigger if I create a new rule using the AWS Console, but for whatever reason it's not able to successfully trigger using serverless/CloudFormation.
Every post I seem to read about this topic makes mention of the same thing - that is to have the permission set on your Lambda, but I've done that and it's still not working. Does anyone have any idea what could be the reason why it's not triggering?
hard one to spot, but since i was using step functions
detail:
state:
- "FAILED"
- "TIMED_OUT"
- "ABORTED"
should be
detail:
status:
- "FAILED"
- "TIMED_OUT"
- "ABORTED"
Related
Using Serverless Framework, how can I make my Lambda function depend on an SQS queue from the resources section as it is the trigger for the function itself?
In my serverless.yaml, I am defining a new queue and Lambda function.
Then, I want to use the queue as an event source (trigger) for my Lambda function.
I do that by creating the queue ARN manually:
functions:
consumer:
handler: App\Service\Consumer
events:
- sqs:
arn:
Fn::Join:
- ':'
- arn:aws:sqs
- Ref: AWS::Region
- Ref: AWS::AccountId
- ${opt:stage}-skill-assigner
And creating the queue in resources:
resources:
Resources:
SkillAssignerQueue:
Type: AWS::SQS::Queue
Properties:
QueueName: ${opt:stage}-skill-assigner
This works fine if I create the queue in a deployment prior to using it as a function trigger.
But if I try to deploy both of them, it fails with this error when it tries to create the event source mapping:
Invalid request provided: Error occurred while ReceiveMessage. SQS Error Code: AWS.SimpleQueueService.NonExistentQueue. SQS Error Message: The specified queue does not exist for this wsdl version.
Fn::Join enables string concatenation which doesn't inform the Serverless Framework (SF) about the dependency of the function on the queue.
We visually can see that but it needs to be done declaratively.
To make this link obvious to SF, use Fn::GetAtt: instead.
It will inform Serverless Framework about the dependency of the Lambda function on the SQS queue.
This should work:
functions:
consumer:
handler: App\Service\Consumer
events:
- sqs:
arn:
Fn::GetAtt:
- SkillAssignerQueue
- Arn
resources:
Resources:
SkillAssignerQueue:
Type: AWS::SQS::Queue
Properties:
QueueName: ${opt:stage}-skill-assigner
The Serveless Framework can automatically create the queue for you. No need to define it in resources
I've defined an SNS topic, an SQS queue, and an SNS subscription resource in a Cloudformation stack. All three are in the same stack, same region, and same AWS account.
Resources:
SqsQueue:
Type: AWS::SQS::Queue
Properties:
QueueName: 'some-queue'
SnsTopic:
Type: AWS::SNS::Topic
Properties:
TopicName: 'some-topic'
SnsSubscription:
Type: AWS::SNS::Subscription
Properties:
Endpoint: !GetAtt [SqsQueue, Arn]
Protocol: sqs
TopicArn: !Ref SnsTopic
When I run the stack, all three resources are created successfully, but when I publish a message from SNS, it's never received by the SQS queue.
I've been following this link (https://aws.amazon.com/premiumsupport/knowledge-center/sqs-sns-subscribe-cloudformation/) and to my knowledge I've done everything I've needed to. What else am I missing?
Thanks!
Additional info
If I delete the subscription that Cloudformation created via the console and then create a new one via the console, messages are published fine. So it must be something incorrect about the subscription.
I used the AWS CLI to compare the properties of the subscription created by the Cloudformation template to the one created by the console. They are the exact same.
You need to add a policy to allow the SNS topic to publish to your queue. Something like this:
SnsToQueuePolicy:
Type: AWS::SQS::QueuePolicy
Properties:
Queues:
- !Ref SqsQueue
PolicyDocument:
Version: '2012-10-17'
Statement:
- Sid: allow-sns-messages
Effect: Allow
Principal: '*'
Resource: !GetAtt SqsQueue.Arn
Action: SQS:SendMessage,
Condition:
ArnEquals:
aws:SourceArn: !Ref SnsTopic
I am trying to create an AWS SQS queue using the serverless framwework,
But I am getting the following error on deploying the severless.yaml
The provided execution role does not have permissions to call SendMessage on SQS
The issue is the IAM role is created by serverless framework and I have no control of what permissions the framework adds to the role,
Ideally, if the function trigger is an SQS, or needs a DLQ configured,
I was hoping the framework would add Send and Receive message permissions to the role, but I guess it did not
Serveless.yaml -
service: dlq
provider:
name: aws
runtime: nodejs12.x
profile: csStage
region: ap-southeast-1
plugins:
- serverless-plugin-lambda-dead-letter
functions:
dlqFunction:
handler: handler.hello
deadLetter:
sqs: dl-queue
You have complete control over the permissions added to that role. You can add an iamRoleStatements section to your serverless.yml file under provider that describes the permissions you wish to apply to the role applied to functions. It would look something like:
provider:
iamRoleStatements:
- Effect: Allow
Action:
- "sqs:SendMessage"
Resource:
- arn:aws:sqs:region:accountid:queueid
You can find out more in the official documentation here: https://www.serverless.com/framework/docs/providers/aws/guide/iam/#iam/
You can use iamRoleStatements to give that permission to your Lambda function. The following template worked for me:
service: dlq
provider:
name: aws
runtime: nodejs12.x
profile: csStage
region: ap-southeast-1
iamRoleStatements:
- Effect: Allow
Action:
- sqs:SendMessage
Resource: !GetAtt DLQ.Arn
plugins:
- serverless-plugin-lambda-dead-letter
functions:
dlqFunction:
handler: handler.hello
deadLetter:
targetArn:
GetResourceArn: DLQ
resources:
Resources:
DLQ:
Type: AWS::SQS::Queue
Properties:
QueueName: dl-queue
I have to make an HTTP request that takes a long time to receive a response. I don't want AWS Lambda to make this request as I will be charged for the time it is waiting for the response. Is there any way to use AWS Lambda to handle the response without being charged while waiting?
Based on my comment, I would recommend hitting the long poll endpoint often as opposed to staying connected for long periods of time. You can use a CloudWatch Rule to trigger the lambda function every 5 minutes (or whatever interval you choose). You can give the Lambda a short timeout, say 5-10 seconds, which should prevent it from running for too long. I'm assuming the long poll endpoint will guarantee delivery at least once.
Here is some CloudFormation YAML to get you started on the setup. Far from complete, but should get you on the right track.
Description: Automatically hit long poll endpoint
Resources:
#################################################
# IAM Role for Lambda
#################################################
ROLELAMBDADEFAULT:
Type: AWS::IAM::Role
Properties:
RoleName: your-lambda-default
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- edgelambda.amazonaws.com
- lambda.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole
- arn:aws:iam::aws:policy/service-role/AWSLambdaRole
Policies: []
#################################################
# Lambda function
#################################################
LFUNC:
Type: AWS::Lambda::Function
Properties:
Code:
S3Bucket: bucket-with-code
S3Key: code.zip
Description: Some function name
FunctionName: my-function-name
Handler: index.handler
MemorySize: 256
Role: !GetAtt ROLELAMBDADEFAULT.Arn
#choose your runtime here
Runtime: nodejs8.10
Timeout: 6
#################################################
# Rule to trigger the lambda
#################################################
RULE1:
Type: AWS::Events::Rule
Properties:
Name: custom-trigger
Description: Trigger my lambda
ScheduleExpression: rate(5 minutes)
State: ENABLED
Targets:
- Arn: !GetAtt LFUNC.Arn
Id: uniqueid1
I followed this tutorial to setup an AWS Lambda function that is invoked upon an upload to S3 and populates DynamoDB.
I'm trying to achieve the same with AWS SAM for which I need to define a template.yaml file with the configuration information. I keep getting this error when deploying with Cloudformation -
Failed to create the changeset: Waiter ChangeSetCreateComplete failed: Waiter encountered a terminal failure state Status: FAILED. Reason: Circular dependency between resources: [LambdaPerm]
I could not find a lot of information on this so I'm struggling to debug. What's causing this error and how can I resolve this? Here's my template configuration -
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
Gradebook:
Type: AWS::Serverless::Function
Properties:
FunctionName: LambdaGradebookServerless
Handler: serverless.LambdaGradebook
Runtime: java8
CodeUri: ./target/serverless-0.0.1-SNAPSHOT.jar
Role: arn:aws:iam::xxxxxxxxxxxx:role/lambda-s3-execution-role
LambdaPerm:
Type: AWS::Lambda::Permission
Properties:
Action: lambda:InvokeFunction
FunctionName:
Ref: Gradebook
Principal: s3.amazonaws.com
SourceAccount:
Ref: AWS::xxxxxxxxxxxx
SourceArn:
Fn::Join:
- ':'
- - arn
- aws
- s3
- ''
- ''
- Ref: gradebookBucket
gradebookBucket:
Type: AWS::S3::Bucket
Properties:
Bucket: gradebook-lambda
NotificationConfiguration:
LambdaConfigurations:
- Event: s3:ObjectCreated:*
Function:
Ref: Gradebook
To avoid this circular dependency, create the S3 bucket and the Lambda function independently, then update the stack with an S3 notification configuration.
If anyone comes searching for this, I wrote a blog to address this scenario.
https://aws.amazon.com/blogs/mt/resolving-circular-dependency-in-provisioning-of-amazon-s3-buckets-with-aws-lambda-event-notifications/
The implementation approach is similar to what jarmod describes in the accepted answer. The event notification is setup later using a CloudFormation custom resource.
I got the circular dependency error and it turns out it was a missing parameter that I was referencing from the resource on the CloudFormation template!.