I'm using the EventBridge to trigger a step function. My EventBridge rule in the CloudFormation template looks as follows:
JobStepFunctionTrigger:
Type: AWS::Events::Rule
Properties:
EventBusName: !GetAtt JobTaskEventBus.Name
Name: !Sub ${DeploymentName}-new-job-created
State: ENABLED
EventPattern:
source:
- !Sub ${DeploymentName}-my-service
detail-type:
- 'NEW_JOB'
Targets:
- Arn: !GetAtt JobOrchestrator.Arn
Id: !GetAtt JobOrchestrator.Name
RoleArn: !Ref MyAwesomeRole
Unfortunately the step function "execution name" is randomly generated in this case making it very difficult to link the specific event to the specific step function execution. in my event I have a property $.detail.id and $.detail.state I would love to be able to use these, to issue the step function execution name in the format ${detail.id}_${detail.state}_someRandomValueToGuaranteeNameUniqueness, but reading the docs about the rule targets I don't see how this would work...
This isn't possible today. The simplest workaround is to redirect to an known API Gateway endpoint[1], and then use API-Gateway's transformations to then trigger the right step-functions[2].
I've got limited knowledge on how API/GW transformations work, so personally prefer using a lambda to call the right step-function. This could be your back up option as well.
[1] https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-api-gateway-target.html
[2] https://docs.aws.amazon.com/apigateway/latest/developerguide/rest-api-data-transformations.html
Related
I'd like to expose the TopicArn Value (referenced in the outputs section at the bottom of my code snippet) of my SNStopic via Cloudformation template in the outputs tab of my stack in a similar manner to the way it's exposed in the resources when I create an SNStopic through the service catalog. I tried to access it by referencing it in the outputs section of my yaml script using dot notation but have been unsuccessful thus far. How might I be able to do so? I'm looking to do this so others using my script in the future won't have to go searching for the TopicArn in another place in order to subscribe to it.
Another important thing to note is that the provisioned product id below, under the properties section of the resources code block generates an SNSTopic.
Resources:
LabTrainingSnsTopic:
Type: "AWS::ServiceCatalog::CloudFormationProvisionedProduct"
Properties:
ProductId: prod-4iafsjovqrsrm # Sns Topic
ProvisioningArtifactName: "v1.1" # Must be an actual version number.
ProvisionedProductName: !Ref ProvisionedProductName
...
Outputs:
AccountID:
Description: The account in which this was built.
Value: !Ref 'AWS::AccountId'
TopicArn:
Description: Arn of the topic we created
Value: !GetAtt LabTrainingHigSnsTopic.ProvisionedProductName.Resources.SNSTopic
service catalog screenshot
cloudformation screenshot
I've got a serverless file which creates an eventbridge rule on the default event bus:
StepFunctionErrorEvent:
Type: AWS::Events::Rule
Properties:
Name: ${self:custom.resourcePrefix}-step-function-error-event-rule
Description: Event bus rule coordinating what targets receive Step Function error events
EventPattern:
source:
- "aws.states"
"detail-type":
- "Step Functions Execution Status Change"
detail:
state:
- "FAILED"
- "TIMED_OUT"
- "ABORTED"
Targets:
- Arn: ${cf:${self:custom.resourcePrefix}-service-internal-slack-integration.PostSlackMessageLambdaArn}
Id: "ErrorSlackMessage"
DeadLetterConfig:
Arn: !GetAtt DefaultErrorTargetDLQ.Arn
DefaultErrorTargetDLQ:
Type: AWS::SQS::Queue
Properties:
QueueName: ${self:custom.resourcePrefix}-DefaultErrorTargetDL
And in a seperate serverless file which also gets deployed I'm adding the following Lambda permission to pl-us-east-2-pilot-post-slack-message:
resources:
Resources:
TriggerPostSlackMessageLambda:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !GetAtt PostSlackMessageLambdaFunction.Arn
Action: lambda:InvokeFunction
Principal: events.amazonaws.com
SourceArn: !Sub arn:aws:events:${AWS::Region}:${AWS::AccountId}:rule/pl-us-east-2-pilot-step-function-error-event-rule
However despite pl-us-east-2-pilot-post-slack-message lambda having the above listed as a permission under 'Resource based policy' (in the Lambda console) the EventBridge rule does not trigger when there is a Lambda failure. It does trigger if I create a new rule using the AWS Console, but for whatever reason it's not able to successfully trigger using serverless/CloudFormation.
Every post I seem to read about this topic makes mention of the same thing - that is to have the permission set on your Lambda, but I've done that and it's still not working. Does anyone have any idea what could be the reason why it's not triggering?
hard one to spot, but since i was using step functions
detail:
state:
- "FAILED"
- "TIMED_OUT"
- "ABORTED"
should be
detail:
status:
- "FAILED"
- "TIMED_OUT"
- "ABORTED"
We are building a serverless app using aws and we want to enable lambda warmers only on the production environment.
Our cloudformation parameters:
Parameters:
Environment:
Description: Environment name
Type: String
EnableWarmer:
Description: Flag to enable/disable warmup Events
Default: DISABLED
Type: String
Our lambdas yaml file looks like this:
MyLambda:
Type: AWS::Serverless::Function
Properties:
FunctionName: !Sub '${Environment}-my-lambda'
CodeUri: src
Handler: lambda.handler
Runtime: nodejs12.x
MemorySize: 128
Timeout: 100
Description: Creates a new something
Layers:
- !Ref InternalDependencyLayer
- !Ref ExternalDependencyLayer
Role: !Ref LambdaRoleArn
Events:
Api:
Type: Api
Properties:
Path: /url
Method: POST
RestApiId: !Ref ApiGateway
WarmingSchedule:
Type: Schedule
Properties:
Enabled: !Ref EnableWarmer
Schedule: rate(5 minutes)
Input: '{ "warmer":true, "concurrency": 2 }'
And then we deploy the dev environment with these params:
- Key: Environment
Value: dev
- Key: EnableWarmer
Value: DISABLED
Similarly for the production environment we deploy with these params:
- Key: Environment
Value: production
- Key: EnableWarmer
Value: ENABLED
According to aws documentation parameters can't be of type boolean which is the required type of the enabled attribute of the schedule event.
Fortunately amazon states:
Enabled Indicates whether the rule is enabled.
To disable the rule, set this property to False.
Type: Boolean
Required: No
AWS CloudFormation compatibility: This property is similar to the
State property of an AWS::Events::Rule resource. If this property is
set to True then AWS SAM passes ENABLED, otherwise it passes DISABLED
https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-property-function-schedule.html
However when we deploy to the dev environment the warmers are enabled.
The problem with the Enabled property here seems to be that it does not support using CloudFormation's If function and a Condition to do something like that:
WarmingSchedule:
Enabled: !If [WarmerCondition, true, false]
(At least I couldn't figure it out, but maybe I just did something wrong) This is similar to what #raul-barreto suggests with !Equals. However, in my tests it always sets the rule's status to ENABLED.
It seems like there are two options left what you can do:
a) (Not recommended) Copy the CloudFormation code of your Lambda function and have a second one available in your CloudFormation resources. Now set a condition to both of them so you only create one of them, depending on your environment. Note: I don't say this is a good approach but it'd be possible. Problems are you have to maintain the definition of both Lambda functions and if you repeat this approach, you'll faster reach CloudFormations template size limits.
b) If you want to make sure you have a minimum number of instances available to your function (at least that's what I interpret from your question), consider using AWS Lambda's Provisioned Concurrency feature instead. You can define it like this in your AWS Lambda function using SAM:
MyLambda:
Type: AWS::Serverless::Function
Properties:
AutoPublishAlias: 'LATEST' #Setting AutoPublishAlias is important, otherwise you can not use provisioned concurrency
ProvisionedConcurrencyConfig:
ProvisionedConcurrentExecutions: 2
This snippet will provision concurrency for the function's alias 'LATEST' which points to the latest version of your function. Read more about AutoPublishAlias in SAM (Note: the link's content is written in the context of traffic shifting but it explains what the property does). It makes sure that you always have 2 instances of your Lambda functions available.
The accepted solution works, but as stated in comments, it is a bit difficult to control the enabling and disabling of the event. The following seems to work in a more robust way.
Events:
Scheduler:
Type: Schedule
Properties:
Schedule: !If [EnableWarmer, "rate(5 minutes)",
"cron(00 00 01 01 ? 1970)"]
According to the documentation, Enabled must a Boolean variable.
You can still have a String parameter and convert it to Boolean inside the CloudFormation.
WarmingSchedule:
Type: Schedule
Properties:
Enabled: !Equals [!Ref EnableWarmer, ENABLED]
Schedule: rate(5 minutes)
Input: '{ "warmer":true, "concurrency": 2 }'
This way you still can send ENABLED or DISABLED as a parameter but the input of WarmingSchedule.Enabled will be a Boolean.
I can't seem to get Ref or Fn:GetAtt to return a valid value for use with setting up a resource.
serverless.yml
...etc...
functions:
bearerTokenAuthentication:
handler: app.bearerTokenAuthentication
name: ${self:service}-auth-bearer
resources:
- ${file(./serverless_resources.yml)}
serverless_resources.yml
Resources:
ApiGateway:
Type: AWS::ApiGateway::RestApi
Properties:
Name: restapi-${self:provider.stage}
Description: Endpoints
ApiKeySourceType: HEADER # (to read the API key from the X-API-Key header of a request)
ApiGatewayBearerAuthorizer:
Type: AWS::ApiGateway::Authorizer
Properties:
Type: token
IdentitySource: method.request.header.Authorization
Name: BearerAuthorization
AuthorizerResultTtlInSeconds: 300
AuthorizerUri: !Join #arn:aws:apigateway:${self:provider.region}:lambda:path/${self:functions.bearerTokenAuthentication.name}
- ''
- - 'arn:aws:apigateway:'
- !Ref 'AWS::Region'
- ':lambda:path/2015-03-31/functions/'
- !GetAtt
- bearerTokenAuthentication # also tried !Ref bearerTokenAuthentication and '${self:functions.bearerTokenAuthentication.name}'
- Arn
- /invocations
RestApiId: !Ref ApiGateway
No matter what I do, GetAtt cannot find the ARN for the Lambda function declared in bearerTokenAuthentication. I just keep getting this error:
Error: The CloudFormation template is invalid: Template error: instance of Fn::GetAtt references undefined resource bearerTokenAuthentication
... or if trying Ref ...
Error: The CloudFormation template is invalid: Template format error: Unresolved resource dependencies [bearerTokenAuthentication] in the Resources block of the template
Is it possible to reference Lambda ARNs from the resource section? It seems by the error messages it is looking for "resource" names. I always thought the lambda function declaration was also considered a resource (besides the obvious Resources: block of course), perhaps I am misunderstanding something.
I figured it out. I had a NodeJS project and was using the "serverless" command line (sls) to deploy using serverless.yml. It turns out it creates a .serverless sub-directroy with some files in it. One of them is a compiled template for AWS Cloud Formation called cloudformation-template-update-stack.json. It appears that the utility likes to mangle the names by making the first character uppercase and adding "LambdaFunction" to all the function names (for whatever reason). In this case, bearerTokenAuthentication was renamed to BearerTokenAuthenticationLambdaFunction (the actual resource name). After looking into the compiled template it all became clear. The utility also seems to figure out the dependencies as well, which was good to know. This was the final result:
AuthorizerUri: !Join
- ''
- - 'arn:aws:apigateway:'
- !Ref 'AWS::Region'
- ':lambda:path/2015-03-31/functions/'
- !GetAtt [ BearerTokenAuthenticationLambdaFunction, Arn ]
- '/invocations'
Other "Gotchas":
DO NOT define the AWS::ApiGateway::RestApi resource (like I did in my question) if you are also using event mappings with the functions, otherwise you will get 2 APIs created. event entries automatically cause an API to be created called "ApiGatewayRestApi" - which is the resource name generated by the sls utility. The last line of the last file was changed to this:
RestApiId: !Ref ApiGatewayRestApi
And my ApiGateway: section was removed.
Credit goes to this post which helped make it more clear to me what was really going on: https://forum.serverless.com/t/fixed-how-do-i-get-reference-api-gateway-restapi-id-in-serverless-yml/3397/5
Previous Answer:
I found another way as well. This is what I resorted to doing until I found the proper (shorter) way. I was able to pull the lambda name and manually stitch together the required URI:
AuthorizerUri: !Join
- ''
- - 'arn:aws:apigateway:'
- !Ref 'AWS::Region'
- ':lambda:path/2015-03-31/functions/arn:aws:lambda:'
- !Ref 'AWS::Region'
- ':'
- !Ref 'AWS::AccountId'
- ':function:'
- '${self:functions.bearerTokenAuthentication.name}'
- '/invocations'
I hope that helps save someone some time trying to understand the complicated .yml file. I also cannot understand why it is so hard to make it simple to understand. All someone had to do is say (for me) was "sls takes a 'serverless.yml' file, and optional include files (such as declarations specific to the cloud system itself, like AWS Cloud Formation), and generates a template JSON file that is used by the target cloud services system to deploy your solution. Also, the names you give may get mangled, so check the template." I'm also surprised that no one has created an editor to make all this easier by now - perhaps something I'll look into myself one day. ;)
You can always go to the deployed lambda and look for the aws:cloudformation:logical-id tag. That way you get the logical ID you should be using in your serverless.yaml. (Don't like this behind-the-scenes names tricks either...)
This is what I am trying to do:
LambdaInvokePermission:
Type: AWS::Lambda::Permission
Properties:
--snip--
SourceArn: !Sub arn:aws:events:${AWS::Region}:${AWS::AccountId}:rule/*-${Environment}
However it seems to dislike the * syntax.
I have tried just SourceArn: !Sub arn:aws:events:${AWS::Region}:${AWS::AccountId}:rule/* and SourceArn: '*' but these also give failure messages like
The rule * could not be found.
Does anyone know the correct syntax for this?
You need to specify the exact name of the cloudwatch rule that will trigger this lambda function.
Example: arn:aws:events:us-east-1:123456789012:rule/my-scheduled-rule.
The SourceArn property of AWS::Lambda::Permission expects a String value: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-permission.html#cfn-lambda-permission-sourcearn.
You cannot have multiple cloudwatch rules as the SourceArn(not a list) in a single AWS::Lambda::Permission block with a wildcard '*'. To have multiple cloudwatch rules trigger the same lambda function, you will need to add another AWS::Lambda::Permission block in your cloudformation template.