I'm trying to deploy an AWS API Gateway and a Lambda function using Gitlab CICD pipeline using sam template. Since I want individual stacks for each environment, the Path property under Events of the Lambda function needs to change with each environment. Below is my code for implementing this:
Mappings:
StackEnv:
stack-dev:
envm: "dev"
stack-qa:
envm: "qa"
Resources:
MyFunction:
Type: AWS::Serverless::Function
Properties:
Events:
MyAPI:
Type: Api
Properties:
Path:
Fn::Sub:
- "/${path}"
- path: !FindInMap [StackEnv, !Ref AWS::StackName, envm]
However, while running the pipeline, an error is returned:
[InvalidResourceException('MyFunction', "Event with id [MyAPI] is invalid. Api Event must have a String specified for 'Path'.")]
Is this behaviour expected while passing in values using !FindInMap? Is there any other way of dynamically passing in the value for Path?
I'm using the EventBridge to trigger a step function. My EventBridge rule in the CloudFormation template looks as follows:
JobStepFunctionTrigger:
Type: AWS::Events::Rule
Properties:
EventBusName: !GetAtt JobTaskEventBus.Name
Name: !Sub ${DeploymentName}-new-job-created
State: ENABLED
EventPattern:
source:
- !Sub ${DeploymentName}-my-service
detail-type:
- 'NEW_JOB'
Targets:
- Arn: !GetAtt JobOrchestrator.Arn
Id: !GetAtt JobOrchestrator.Name
RoleArn: !Ref MyAwesomeRole
Unfortunately the step function "execution name" is randomly generated in this case making it very difficult to link the specific event to the specific step function execution. in my event I have a property $.detail.id and $.detail.state I would love to be able to use these, to issue the step function execution name in the format ${detail.id}_${detail.state}_someRandomValueToGuaranteeNameUniqueness, but reading the docs about the rule targets I don't see how this would work...
This isn't possible today. The simplest workaround is to redirect to an known API Gateway endpoint[1], and then use API-Gateway's transformations to then trigger the right step-functions[2].
I've got limited knowledge on how API/GW transformations work, so personally prefer using a lambda to call the right step-function. This could be your back up option as well.
[1] https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-api-gateway-target.html
[2] https://docs.aws.amazon.com/apigateway/latest/developerguide/rest-api-data-transformations.html
I have a lambda function created using SAM template which has an event trigger scheduled every 5 mins.
When the lambda get created for first time the event is enabled. But, when someone disables the trigger manually, the rerunning the stack does not enable it again.
Native cloud formation has an attribute called State in AWS::Events::Rule. But this is not supported in SAM Function's Events property. It is to be noted that this Events property gets translated to AWS::Events::Rule by SAM engine.
I tried adding the same State attribute in SAM but that doen't work.
Now the question is really how do I make sure Event is always enabled when SAM is used.
This there a hack available.
Sample code:
MyUpdater:
Type: 'AWS::Serverless::Function'
Properties:
Handler: myupdater.lambda_handler
Runtime: python3.6
FunctionName: "myupdater"
CodeUri: ./code
Description: Sample updater lambda
MemorySize: 128
Timeout: 60
Role: !ImportValue myIamRole
KmsKeyArn: !ImportValue myKeyArn
Events:
Timer:
Type: Schedule
Properties:
Schedule: rate(5 minutes)
Thanks in advance
This is not possible today. We are considering this feature (CFN properties pass-through) for the future.
I'm finding it hard to understand the difference between SAM template and Cloudformation template. I know that SAM template can be used to define Serverless Applications like Lambda, but how does that make it different from Cloudformation template? Is the syntax different? I can still specify the Lambda definitions in cloudformation template. So, my question is why should I care about SAM? Won't knowing about just cloud formation template be sufficient?
From CloudFormation's perspective, SAM is a transform. Meaning: SAM templates are syntactically equivalent, but they allow you to define your serverless app with more brevity. The SAM template eventually gets expanded into full CFN behind the scenes. If you already know CFN, but want to write less YAML code, SAM may be beneficial to you. The idea is to reduce your effort.
SAM templates are a superset of Cloudformation. Any Cloudformation template can be run through SAM unchanged, and it will work. SAM supports all the types available in Cloudformation templates, so you can think of SAM as "CloudFormation++".
However, SAM also gives you additional "transforms" that allow you to define certain concepts succinctly, and SAM will figure out what you mean and fill in the the missing pieces to create a full, expanded, legal Cloudformation template.
Example: For SAM (and Serverless Framework) users, who deal mostly in Lambda functions, one of the more most useful transforms is the Events property on the Lambda function -- SAM will add all the objects needs to access that function through an API path in API Gateway.
Resources:
HelloWorldFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: HelloWorldFunction
Handler: app.lambdaHandler
Runtime: nodejs12.x
Events: # <--- "Events" property is not a real Cloudformation Lambda property
HelloWorld:
Type: Api
Properties:
Path: /hello
Method: get
The SAM template snippet shown above gets transformed/expanded into several API Gateway objects (a RestApi, a deployment, and a stage). The AWS::Serverless::Function type used in this snippet is not a real Cloudformation type -- you won't find it in the docs. SAM expands it into a Cloudformation template containing a AWS::Lambda::Function object and several different AWS::ApiGateway::* objects that Cloudformation understands.
To give you an idea of how much manual coding this saves you, here's what the expanded version of the above SAM template looks like as a full Cloudformation template:
Resources:
HelloWorldFunctionHelloWorldPermissionProd:
Type: AWS::Lambda::Permission
Properties:
Action: lambda:InvokeFunction
Principal: apigateway.amazonaws.com
FunctionName:
Ref: HelloWorldFunction
SourceArn:
Fn::Sub:
- arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${__ApiId__}/${__Stage__}/GET/hello
- __Stage__: "*"
__ApiId__:
Ref: ServerlessRestApi
HelloWorldFunctionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Action:
- sts:AssumeRole
Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
Tags:
- Value: SAM
Key: lambda:createdBy
ServerlessRestApiProdStage:
Type: AWS::ApiGateway::Stage
Properties:
DeploymentId:
Ref: ServerlessRestApiDeployment_NNN
RestApiId:
Ref: ServerlessRestApi
StageName: Prod
ServerlessRestApiDeployment_NNN:
Type: AWS::ApiGateway::Deployment
Properties:
RestApiId:
Ref: ServerlessRestApi
Description: 'RestApi deployment id: ???'
StageName: Stage
ServerlessRestApi:
Type: AWS::ApiGateway::RestApi
Properties:
Body:
info:
version: '1.0'
title:
Ref: AWS::StackName
paths:
"/hello":
get:
x-amazon-apigateway-integration:
httpMethod: POST
type: aws_proxy
uri:
Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${HelloWorldFunction.Arn}/invocations
responses: {}
swagger: '2.0'
HelloWorldFunction:
Type: AWS::Lambda::Function
Properties:
Code:
S3Bucket: aws-sam-cli-managed-default-samclisourcebucket-???
S3Key: temp/???
Tags:
- Value: SAM
Key: lambda:createdBy
Handler: app.lambdaHandler
Role:
Fn::GetAtt:
- HelloWorldFunctionRole
- Arn
Timeout: 3
Runtime: nodejs12.x
Previously, if you were authoring pure Cloudformation, you would have had to code all this by hand, over and over, for each API Gateway endpoint that you wanted to create. Now, with a SAM template, you define the API as an "Event" property of the Lambda function, and SAM (or Serverless Framework) takes care of the drudgery.
In the old days, when we had to do all this by hand, it totally sucked. But now, everything is glorious again.
Like #Luis Colon said, SAM is a transform. What that means, is that at the top of a SAM Template there is a Transform statement that lets CloudFormation know to run an intrinsic function, Transform, on this SAM template to turn it into a CloudFormation template. So, all SAM Templates will eventually be converted into CF templates, but for the end-user in most cases it is easier to just use the SAM template. For instance, for a simple application with Lambdas triggered by a new API you're creating, the SAM template will let you accomplish this in fewer lines than CloudFormation.
To extend this, the Serverless Framework behaves similarly. Serverless is designed to work across platforms (AWS, Azure, etc.). It's syntax looks very similar to SAM, and it too converts the template into the target platform's (ie. AWS) fuller version of the template (ie. CloudFormation template).
You can imagine SAM as an extended form of CloudFormation. SAM makes Serverless/Lambda deployments easier.
Even CloudFormation can deploy lambda scripts using inline scripts but it has a limitation of 4096 characters and you cannot pack custom dependencies, python libraries.
So to make Lambda/Serverless deployments easy SAM is used. SAM is a CLI tool. You cannot find SAM in AWS Console.
In case of python deployment, sam will read the requirements.txt file build a package, and will deploy the package when you wish to sam deploy
So at the end of the day you can write as much lengthy Lambda Code, use as many libraries you want and even import your custom libraries i.e. complete flexibility.
We use Cloud Formation for define a bunch of Lambda functions:
AWSTemplateFormatVersion: '2010-09-09'
Transform:
- 'AWS::Serverless-2016-10-31'
Resources:
MyLambda:
Type: 'AWS::Serverless::Function'
Properties:
Handler: com.handler::MyLambda
Runtime: java8
CodeUri: .
Description: Some desc
MemorySize: 512
Timeout: 15
Role: !Ref LambdaRole
FunctionName: MyLambda
Events:
MyLambdaEvt:
Type: Api
Properties:
RestApiId: !Ref MyApiDef
Path: /lambda/my
Method: get
MyApiDef:
Type: AWS::Serverless::Api
Properties:
DefinitionUri: s3://a-bucket/api-gateway.yml
StageName: prod
Outputs:
ApiUrl:
Description: URL of your API endpoint
Value: !Join
- ''
- - https://
- !Ref MyApiDef
- '.execute-api.'
- !Ref 'AWS::Region'
- '.amazonaws.com/prod'
A CodePipeline generate a changeset and execute it.
In this way all the Lambda function are correctly updated but the API Gateway endpoint are not update correctly and we need to import and deploy the YML in s3://a-bucket/api-gateway.yml manually.
Why the API doesn't update (an educated guess)
In order for a change to be added to a a change set, CloudFormation has to detect a change. If the only thing that changes (for MyApiDef) between deployments is the contents of the .yaml file out on S3, CloudFormation isn't going to detect a change that it needs to add to the change set.
If this API definition lived in the CF template, rather than a file on S3, CF would (obviously) detect every change and update the API for you.
Since the definition lives in S3, and the file name hasn't changed, no change is detected, so nothing gets updated.
Possible work arounds
You have to convince CloudFormation that something has changed with your API definition. These two things worked for me:
Updating the MyApiDef key itself each run works. (MyApiDefv2,
MyApiDefv3, etc)
Updating the DefinitionUri works. (i.e. version the
filename in S3).
Neither of these is great, but appending a version to the filename in S3 seems more reasonable than the other option.
There are probably other ways to convince CloudFormation a change has taken place. Notably, I could not get Variables to work for this purpose.