AWS StateMachine for Lambda, cloud formation syntax - amazon-web-services

I'm trying to come up with a CloudFormation template that includes
API Gateway
Invokes a StateMachine through the API Gateway
StateMachine in turn contains a lambda function
In essence what I'm try to do is the following
https://docs.aws.amazon.com/step-functions/latest/dg/tutorial-api-gateway.html
However I'm stuck in coming up with the Cloud Formation Template (.yaml) that will deploy this. So far this is what I have
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
Post:
Type: AWS::Serverless::Function
Properties:
FunctionName: UserBase-fnUsers
Handler: UsersHandler.getUsers
Runtime: nodejs6.10
Policies: [AmazonDynamoDBReadOnlyAccess, AmazonS3ReadOnlyAccess]
Environment:
Variables:
S3_BUCKET: UserBase-Users-bucket
UsersTable: UserBase-Users-tblUsers
Events:
GetUsers:
Type: Api
Properties:
Path: /UserBase/Users
Method: post
Options:
Type: AWS::Serverless::Function
Properties:
FunctionName: UserBase-fnUsers-Options
Handler: UsersHandler.getOptions
Runtime: nodejs6.10
Events:
GetOptions:
Type: Api
Properties:
Path: /UserBase/Users
Method: options
UsersTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: UserBase-Users-tblUsers
AttributeDefinitions:
- AttributeName: Id
AttributeType: S
KeySchema:
- AttributeName: Id
KeyType: HASH
ProvisionedThroughput:
ReadCapacityUnits: 5
WriteCapacityUnits: 5
StreamSpecification:
StreamViewType: KEYS_ONLY
StatesExecutionRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Principal:
Service:
- !Sub states.${AWS::Region}.amazonaws.com
Action: "sts:AssumeRole"
Path: "/"
Policies:
- PolicyName: StatesExecutionPolicy
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- "lambda:InvokeFunction"
Resource: "*"
UpdateShoppingPath:
Type: "AWS::StepFunctions::StateMachine"
Properties:
DefinitionString:
!Sub
- |-
{
"Comment": "State machine to update the shopping path",
"StartAt": "UpdatePath",
"States": {
"UpdatePath": {
"Type": "Task",
"Resource": "${lambdaArn}",
"End": true
}
}
}
- {lambdaArn: !GetAtt [ Post, Arn ]}
RoleArn: !GetAtt [ StatesExecutionRole, Arn ]
UserBaseUsers:
Type: "AWS::ApiGateway::Resource"
I'm stuck with the last piece, basically on how to link the ApiGateway to the StateMachine. On a side note is there any way for me to generate the cloud formation template (.yaml or json) from an existing deployment in AWS?

I'm not an expert with yaml, but I did some configuration with json CloudFormation and as far as I have read it's quite easy to translate.
In the past I've been stuck like you, and here's my post and my solution
What you need to do to start the execution of a Step Functions is to do a HTTP Post to arn:aws:apigateway:${region}:states:action/StartExecution passing as json object [docs]:
{
input: __input__,
stateMachineArn: __arn__
}
In short, in your AWS::ApiGateway::Method, you have to set an HTTP integration to arn:aws:apigateway:${region}:states:action/StartExecution and a requestTemplate that builds the json object I mentioned.
For reference, here my json cloudformation example:
"FooRequest": {
"DependsOn": ["FooStepMachine"],
"Type": "AWS::ApiGateway::Method",
"Properties": {
"HttpMethod": "POST",
"Integration": {
"Type": "AWS",
"Credentials": {
"Fn::GetAtt": ["FooRole",
"Arn"]
},
"IntegrationHttpMethod": "POST",
"Uri": {
"Fn::Join": ["",
["arn:aws:apigateway:",
{
"Ref": "AWS::Region"
},
":states:action/StartExecution"]]
},
"IntegrationResponses": [{
"StatusCode": 200
},
{
"StatusCode": 401
}],
"RequestTemplates": {
"application/json": {
"Fn::Sub": ["{\"input\": \"$util.escapeJavaScript($input.json('$'))\",\"stateMachineArn\": \"${arn}\"}",
{
"arn": {
"Ref": "FooStepMachine"
}
}]
}
}
}
}
}

Related

AWS SAM, how to run a state machine from an api gateway call?

I'm trying to set up a state machine for a workflow, but for the life of me I cannot seem to get it working, here is my SAM template:
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: >
strest
Sample SAM Template for strest
Globals:
Function:
Timeout: 3
Resources:
PublicApi:
Type: AWS::Serverless::Api
Properties:
StageName: Prod
# TracingEnabled: true
DefinitionBody:
swagger: "2.0"
info:
version: "1.1"
title: "StrestApi"
schemes:
- "http"
paths:
/start: # api gateway invokes lambda synchronously, which in turn invokes the stepfunction and waits for its final result
get:
produces:
- "application/json"
responses:
"200":
description: "200 response"
schema:
$ref: "#/definitions/Empty"
headers:
Access-Control-Allow-Headers:
type: "string"
security: []
x-amazon-apigateway-integration:
responses:
default:
statusCode: "200"
headers:
Access-Control-Allow-Headers:
type: "'*'"
httpMethod: GET
type: aws_proxy
uri:
Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${StartFunction.Arn}/invocations
definitions:
Empty:
type: "object"
title: "Empty Schema"
# Role which allows step functions to invoke lambda functions
StatesExecutionRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Principal:
Service:
- !Sub states.${AWS::Region}.amazonaws.com
Action: "sts:AssumeRole"
Path: "/"
Policies:
- PolicyName: StatesExecutionPolicy
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- "lambda:InvokeFunction"
Resource: "*"
# LAMBDAS
StartFunction:
Type: AWS::Serverless::Function
Properties:
Description: Starts the state machine
CodeUri: dist/
Handler: start/start.handler
Runtime: nodejs12.x
Environment:
Variables:
STEP_FUNCTION_ARN: !Ref StepFunctionsStateMachine
Policies:
- Version: "2012-10-17"
Statement:
- Effect: "Allow" # step function permissions open for now
Action:
- states:*
Resource: "*"
Events:
ExecSFNResource:
Type: Api
Properties:
RestApiId: !Ref PublicApi
Path: /start
Method: GET
ExecutorFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: dist/
Handler: executor/executor.handler
Runtime: nodejs12.x
# Events:
# HelloWorld:
# Type: Api
# Properties:
# Path: /execute
# Method: get
# State machine
StepFunctionsStateMachine:
Type: AWS::StepFunctions::StateMachine
Properties:
RoleArn: !GetAtt [StatesExecutionRole, Arn]
DefinitionString: !Sub |-
{
"StartAt": "execute",
"Comment": "State machine for executing the strest main loop",
"States": {
"execute": {
"Type": "Task",
"Resource": "${ExecutorFunction.Arn}",
"Comment": "Run the Executor Lambda function",
"End": true
}
}
}
I either start the service by doing sam local start-api or sam local start-lambda.
is there any difference between starting the api with either of these commands?
In the template I pasted, I'm using !Ref to get the state machine ARN, however this is not working, the same string is returned, it works if I change it to !GetAtt StepFunctionsStateMachine.Arn
After changing 2. then I query the /start endpoint, the start lambda function starts running, I get the arn of the state machine, but when I try to start it I get a Service not valid in this context: lambda error (after the marker 2), here is the code for the start function:
import AWS from "aws-sdk";
export async function handler(event: any, context: any) {
let stepFunctionArn = process.env.STEP_FUNCTION_ARN;
console.log("marker0 stepFunctionArn", stepFunctionArn);
let params = {
stateMachineArn: stepFunctionArn!,
name: "Execution lambda " + new Date().toString()
};
console.log("marker 1");
let sf_client = new AWS.StepFunctions();
console.log("marker 2");
let res = await sf_client.startExecution(params).promise();
console.log("marker 3", res);
return {};
}
It's viable to start a step function from a Lambda function, but i think in your case it's a better solution to start it directly from Api Gateway by using the DefinitionBody of API Gateway like this:
/workflow:
post:
x-amazon-apigateway-integration:
credentials:
Fn::GetAtt: [ ApiGatewayStepFunctionsRole, Arn ]
uri:
Fn::Sub: arn:aws:apigateway:${AWS::Region}:states:action/StartExecution
httpMethod: POST
type: aws
responses:
default:
statusCode: 200
responseTemplates:
application/json: |
'{ "executionId": "$input.json('executionArn').split(':').get(7) }'
requestTemplates:
application/json:
Fn::Sub: |-
{
"input": "$util.escapeJavaScript($input.json('$'))",
"name": "$context.requestId",
"stateMachineArn": "${Workflow}"
}
summary: Start workflow instance
responses:
200:
$ref: '#/components/responses/200Execution'
403:
$ref: '#/components/responses/Error'
I have a working example commited in github in https://github.com/jvillane/aws-sam-step-functions-lambda/blob/master/openapi.yaml with an additional method for checking the execution state.

Error parsing parameter '--parameters': Expected: '=', received: 'P'

I am working with aws cli cloudformation. While using the JSON parameters file along with yml template, I keep getting the error. I tried using create stack update stack as well as the change set.
Error parsing parameter '--parameters': Expected: '=', received: 'P' for input:
- ParameterKey: FunctionName
^
ParameterValue: taskaplambda
- ParameterKey: MemorySize
ParameterValue: 512
- ParameterKey: Timeout
ParameterValue: 5
Where my command is:
aws cloudformation update-stack --stack-name apstack --template-body file://templates/cflambdatemplate.yaml --parameters file://params/param.json
And my param.json is:
[
{
"ParameterKey": "FunctionName",
"ParameterValue": "taskaplambda"
},
{
"ParameterKey": "MemorySize",
"ParameterValue": 512
},
{
"ParameterKey": "Timeout",
"ParameterValue": 5
}
]
This is my YAML file
cflambdatemplate.yaml
Transform: AWS::Serverless-2016-10-31
Resources:
tasklambda:
Type: AWS::Serverless::Function
Properties:
FunctionName: !Ref FunctionName
Handler: lambda_function.lambda_handler
MemorySize: !Ref MemorySize
Role:
Fn::GetAtt:
- "tasklambdarole"
- "Arn"
Runtime: python3.7
Timeout: !Ref Timeout
CodeUri:
Bucket: taskapbucket
Key: apbuild/lambda_function.zip
tasklambdarole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: "Allow"
Principal:
Service:
- "lambda.amazonaws.com"
Action:
- "sts:AssumeRole"
Path: "/"
taskPolicies:
Type: "AWS::IAM::Policy"
Properties:
PolicyName: "root"
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: "Allow"
Action: "*"
Resource: "*"
Roles:
-
Ref: "tasklambdarole"
Parameters:
FunctionName:
Type: String
MinLength: '3'
MaxLength: '18'
MemorySize:
Type: Number
MinValue: '128'
MaxValue: '1024'
Timeout:
Type: Number
MinValue: '1'
MaxValue: '15'
I have been trying this with every possibility, but it keeps giving me an error.
Just able to create everything you needed through aws CLI:
I'm using the same yaml file and json template of parameter and not getting any error. Below is the only change that I did in param.json :
[
{
"ParameterKey": "FunctionName",
"ParameterValue": "taskaplambda"
},
{
"ParameterKey": "MemorySize",
"ParameterValue": "512"
},
{
"ParameterKey": "Timeout",
"ParameterValue": "5"
}
]
You need to convert the Number to String, It's because CloudFormation parameter types don't map to JSON types, so the CLI expects everything to be passed as string.

CloudFormation Template is invalid: Template format error: Every Outputs member must contain a Value object

I have an AWS IoT Chat Application whose UI is on React and to do the AWS configuration I have a setup which is executed using "serverless deploy" command. When executed, a serverless.yml gets executed, and it breaks at a point where it throws an error as
CloudFormation Template is invalid: Template format error: Every Outputs member must contain a Value object
the serverless.yml code is given below:
resources:
Resources:
UserTable:
Type: "AWS::DynamoDB::Table"
Properties:
TableName: "IotChatUsers"
AttributeDefinitions:
- AttributeName: identityId
AttributeType: S
KeySchema:
- AttributeName: identityId
KeyType: HASH
ProvisionedThroughput:
ReadCapacityUnits: 5
WriteCapacityUnits: 5
ChatTable:
Type: "AWS::DynamoDB::Table"
Properties:
TableName: "IotChatChats"
AttributeDefinitions:
- AttributeName: name
AttributeType: S
KeySchema:
- AttributeName: name
KeyType: HASH
ProvisionedThroughput:
ReadCapacityUnits: 5
WriteCapacityUnits: 5
ConnectPolicy:
Type: "AWS::IoT::Policy"
Properties:
PolicyName: IotChatConnectPolicy
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Action:
- "iot:Connect"
Resource:
- "*"
PublicSubscribePolicy:
Type: "AWS::IoT::Policy"
Properties:
PolicyName: IotChatPublicSubscribePolicy
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Action:
- "iot:Subscribe"
Resource: { "Fn::Join" : ["",["arn:aws:iot:",{"Ref":"AWS::Region"},":",{"Ref":"AWS::AccountId"},":topicfilter/room/public/*"]] }
PublicReceivePolicy:
Type: "AWS::IoT::Policy"
Properties:
PolicyName: IotChatPublicReceivePolicy
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Action:
- "iot:Receive"
Resource: { "Fn::Join" : ["",["arn:aws:iot:",{"Ref":"AWS::Region"},":",{"Ref":"AWS::AccountId"},":topic/room/public/*"]] }
UserPool:
Type: "AWS::Cognito::UserPool"
Properties:
UserPoolName: iot_chat_api_user_pool
AutoVerifiedAttributes:
- email
MfaConfiguration: OFF
Schema:
- AttributeDataType: String
Name: email
Required: true
ReactAppClient:
Type: AWS::Cognito::UserPoolClient
Properties:
GenerateSecret: false
RefreshTokenValidity: 200
UserPoolId:
Ref: UserPool
IdentityPool:
Type: "AWS::Cognito::IdentityPool"
Properties:
IdentityPoolName: iot_chat_api_identity_pool
AllowUnauthenticatedIdentities: false
CognitoIdentityProviders:
- ClientId:
Ref: ReactAppClient
ProviderName:
Fn::GetAtt: UserPool.ProviderName
SupportedLoginProviders:
graph.facebook.com: ${self:custom.variables.facebook_app_id}
accounts.google.com: ${self:custom.variables.google_app_id}
IdentityPoolAuthRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Principal:
Federated:
- "cognito-identity.amazonaws.com"
Action:
- "sts:AssumeRoleWithWebIdentity"
Condition:
StringEquals:
cognito-identity.amazonaws.com:aud:
Ref: IdentityPool
ForAnyValue:StringLike:
cognito-identity.amazonaws.com:amr: authenticated
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AWSIoTDataAccess
Path: "/"
Policies:
- PolicyName: iot-chat-invoke-api-gateway
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- execute-api:Invoke
Resource: { "Fn::Join" : ["", ["arn:aws:execute-api:",{"Ref":"AWS::Region"},":",{"Ref":"AWS::AccountId"},":",{"Ref":"ApiGatewayRestApi"},"/*"]] }
IdentityPoolRoleAttachment:
Type: AWS::Cognito::IdentityPoolRoleAttachment
Properties:
IdentityPoolId:
Ref: IdentityPool
Roles:
authenticated:
Fn::GetAtt:
- IdentityPoolAuthRole
- Arn
ConfirmUserInvocationPermission:
Type: AWS::Lambda::Permission
Properties:
Action: lambda:InvokeFunction
FunctionName:
Fn::GetAtt: AutoConfirmUserLambdaFunction.Arn
Principal: cognito-idp.amazonaws.com
SourceArn:
Fn::GetAtt: UserPool.Arn
Outputs:
UserPoolId:
Description: "The ID of the user pool that is created."
Value:
Ref: UserPool
ReactAppClientId:
Description: "The ID of the user pool react app client id."
Value:
Ref: ReactAppClient
IdentityPoolId:
Description: "The ID of the identity pool that is created."
Value:
Ref: IdentityPool
AutoConfirmUserFnArn:
Description: "The ARN of the Auto Confirm User Lambda function"
Value:
Fn::GetAtt:
- AutoConfirmUserLambdaFunction
- Arn
FacebookAppId:
Description: "Facebook App Id"
Value: ${self:custom.variables.facebook_app_id}
GoogleAppId:
Description: "Google App Id"
Value: ${self:custom.variables.google_app_id}
I need some insight to figure out what is wrong with the serverless.yml which throws me this validation error.
Environment Information -----------------------------
OS: win32
Node Version: 8.9.1
Serverless Version: 1.25.0
UPDATE:
On parsing the YAML below is the result of Outputs node:
"Outputs": {
"IdentityPoolId": {
"Description": "The ID of the identity pool that is created.",
"Value": {
"Ref": "IdentityPool"
}
},
"FacebookAppId": {
"Description": "Facebook App Id",
"Value": "${self:custom.variables.facebook_app_id}"
},
"ReactAppClientId": {
"Description": "The ID of the user pool react app client id.",
"Value": {
"Ref": "ReactAppClient"
}
},
"GoogleAppId": {
"Description": "Google App Id",
"Value": "${self:custom.variables.google_app_id}"
},
"UserPoolId": {
"Description": "The ID of the user pool that is created.",
"Value": {
"Ref": "UserPool"
}
},
"AutoConfirmUserFnArn": {
"Description": "The ARN of the Auto Confirm User Lambda function",
"Value": {
"Fn::GetAtt": [
"AutoConfirmUserLambdaFunction",
"Arn"
]
}
}
}
Update 2:
This is where the complete application comes from: aws-iot-chat-example
CloudFormation often provides vague or hard to track errors, and never reports errors with line numbers, like many interpreters/compilers/parsers. So tracking them down is often a process of trial and error.
In your case, the error message only mentioned that the error is in the Output section of the template, but it does not mention which value is the problem. You have 6 values in that section.
A good technique for troubleshooting is to remove each of the items one or 2 at a time, and re run the template. Since the output values are just that - only Outputs - they are not needed by this template, but instead expose data to other templates later in the creation process. Just remove them as suggested, and use this technique to isolate the field with the error in the value.
A good sanity check is the remove the entire Outputs section, and confirm that the reset of the template creates as expected.
Once you track down the fields(s) with the problem, you will need to track down the primary issue: Every Outputs member must contain a Value object
To resolve that, track down the object being referenced, and track back to the source resource or resource attribute. For some reason, those references do not refer to a valid object.
I will note that in your comments, you identified two fields that were both causing the error. Both seem to use a variable reference in the form of self:custom.variables.google_app_id - These value are not resolving properly. Check their source as above. I suspect they are not being parsed properly. I do not recognize that construction as valid CloudFormation syntax.

AWS Create Cloudformation log alert for Lambda

I want to create an alert if something goes wrong with Lambda function especially when lambda throws an exception. I am planning to configure SNS topic to send a message if that alert is triggered.
All lambdas are created using CloudFormation scripts, so I am searching for a CloudFormation template to configure alarms on CloudWatch logs. I was not able to find a good/working sample. Sample code below .
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "AWS CloudTrail API Activity Alarm Template for CloudWatch Logs",
"Parameters" : {
"LogGroupName" : {
"Type" : "String",
"Default" : "CloudTrail/DefaultLogGroup",
"Description" : "Enter CloudWatch Logs log group name. Default is CloudTrail/DefaultLogGroup"
},
"Email" : {
"Type" : "String",
"Description" : "Email address to notify when an API activity has triggered an alarm"
}
},
"Resources" : {
"SecurityGroupChangesAlarm": {
"Type": "AWS::CloudWatch::Alarm",
"Properties": {
"AlarmName" : "CloudTrailSecurityGroupChanges",
"AlarmDescription" : "Alarms when an API call is made to create, update or delete a Security Group.",
"AlarmActions" : [{ "Ref" : "AlarmNotificationTopic" }],
"MetricName" : "SecurityGroupEventCount",
"Namespace" : "CloudTrailMetrics",
"ComparisonOperator" : "GreaterThanOrEqualToThreshold",
"EvaluationPeriods" : "1",
"Period" : "300",
"Statistic" : "Sum",
"Threshold" : "1"
}
},
"AlarmNotificationTopic": {
"Type": "AWS::SNS::Topic",
"Properties": {
"Subscription": [
{
"Endpoint": { "Ref": "Email" },
"Protocol": "email"
}
]
}
}
}
}
In order to do this, we need to create a subscription filter on the log group for that lambda with FilterPattern: "Exception"
So whenever there is an Exception word in log message it will trigger a monitor lambda.
Following is a cloudformation template in YAML that I have written
Resources:
LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: 'AllowLambdaAccess'
PolicyDocument:
Version: '2012-10-17'
Statement:
- Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Effect: "Allow"
Resource:
Fn::Join:
- ''
- - 'arn:aws:logs:'
- Ref: AWS::Region
- ':'
- Ref: AWS::AccountId
- ':log-group:/aws/lambda/*'
- Action:
- ec2:DescribeNetworkInterfaces
- ec2:CreateNetworkInterface
- ec2:DeleteNetworkInterface
Effect: "Allow"
Resource: "*"
RoleName: !Sub "${AWS::StackName}-LambdaExecutionRole"
SubscriptionFilter:
Type: "AWS::Logs::SubscriptionFilter"
DependsOn: "LambdaInvokePermission"
Properties:
LogGroupName: !Sub "/aws/lambda/${LogGroupName}"
FilterPattern: "Exception"
DestinationArn:
Fn::GetAtt:
- "LambdaFunction"
- "Arn"
LambdaFunction:
Type: 'AWS::Lambda::Function'
Properties:
Code:
S3Bucket: !Ref S3BucketName
S3Key: !Ref ZipFile
Description: Monitor Lambda Function
Handler: 'index.handler'
MemorySize: 1536
Role: !GetAtt
- LambdaExecutionRole
- Arn
Runtime: nodejs6.10
Environment:
Variables:
SMTP_SERVER: !Ref SMTPServer
SMTP_PORT: !Ref SMTPPort
EMAIL_FROM: !Ref FromEmail
EMAIL_TO: !Ref ToEmail
Timeout: 300
FunctionName: !Sub "${AWS::StackName}-LambdaFunction"
VpcConfig:
SecurityGroupIds: !Split [ ",", !Ref SecurityGroupId ]
SubnetIds: !Split [ ",", !Ref SubnetIds ]
DependsOn:
- LambdaExecutionRole
LambdaInvokePermission:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !Ref "LambdaFunction"
Action: "lambda:InvokeFunction"
Principal: !Sub "logs.${AWS::Region}.amazonaws.com"
SourceArn:
Fn::Join:
- ''
- - 'arn:aws:logs:'
- Ref: AWS::Region
- ':'
- Ref: AWS::AccountId
- !Sub ':log-group:/aws/lambda/${LogGroupName}*'

How to invoke an AWS Step Function using API Gateway

How do I invoke an AWS Step Function using an API Gateway POST request, and the request's JSON payload to the Step Function ?
1. Create your step function
Quite obvious. I guess that if you're reading this you know how to do it.
Otherwise, you can go have a look at the documentation here: What is AWS Step Functions?.
2. Create the IAM Role for your API
It can be for either all Step Functions, or only this one. We'll only cover the first case, as explained in an Amazon tutorial: Creating an API Using API Gateway.
To create the IAM role
Log in to the AWS Identity and Access Management console.
On the Roles page, choose Create New Role.
On the Set Role Name page, type APIGatewayToStepFunctions for Role Name, and then choose Next Step.
On the Select Role Type page, under Select Role Type, select Amazon API Gateway.
On the Attach Policy page, choose Next Step.
On the Review page, note the Role ARN, for example:
arn:aws:iam::123456789012:role/APIGatewayToStepFunctions
Choose Create Role.
To attach a policy to the IAM role
On the Roles page, search for your role by name (APIGatewayToStepFunctions) and then choose the role.
On the Permissions tab, choose Attach Policy.
On the Attach Policy page, search for AWSStepFunctionsFullAccess, choose the policy, and then choose Attach Policy.
3. Setup
3.a If you don't have a JSON payload
As explained by Ka Hou Ieong in How can i call AWS Step Functions by API Gateway?, you can create an AWS Service integration via API Gateway Console, like this:
Integration Type: AWS Service
AWS Service: Step Functions
HTTP method: POST
Action Type: Use action name
Action: StartExecution
Execution role: role to start the execution (the one we just created. Just paste it's ARN)
Headers:
X-Amz-Target -> 'AWSStepFunctions.StartExecution'
Content-Type -> 'application/x-amz-json-1.0'
Body Mapping Templates/Request payload:
{
"input": "string" (optional),
"name": "string" (optional),
"stateMachineArn": "string"
}
3.b If you do have JSON payload to pass as an input
Everything is the same as in 2.a, except for the body mapping template. You have to do is make it into a string. Using $util.escapeJavascript(), like this for example. It will pass your whole request's body as an input to your Step Function
#set($data = $util.escapeJavaScript($input.json('$')))
{
"input": "$data",
"name": "string" (optional),
"stateMachineArn": "string" (required)
}
Notes
stateMachineArn: If you do not want to have to pass the stateMachineArn as part of your requests to API Gateway, you can simply hard-code it inside your Body Mapping Template (see AWS API Gateway with Step Function)
name: Omitting the name property will have API Gateway generate a different one for you at each execution.
Now, this is my first "Answer your own question", so maybe this is not how it's done, but I did spend quite a few hours trying to understand what was wrong with my Mapping Template. Hope this will help save other people's hair and time.
For those ones that are looking a way to directly connect ApiGateway with a Step Functions State Machine using the OpenApi integration and CloudFormation, this is an example of how I managed to make it work:
This is the Visual Workflow I designed (more details in the CloudFormation file) as a proof of concept:
template.yaml
AWSTemplateFormatVersion: 2010-09-09
Transform: 'AWS::Serverless-2016-10-31'
Description: POC Lambda Examples - Step Functions
Parameters:
CorsOrigin:
Description: Header Access-Control-Allow-Origin
Default: "'http://localhost:3000'"
Type: String
CorsMethods:
Description: Header Access-Control-Allow-Headers
Default: "'*'"
Type: String
CorsHeaders:
Description: Header Access-Control-Allow-Headers
Default: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key'"
Type: String
SwaggerS3File:
Description: 'S3 "swagger.yaml" file location'
Default: "./swagger.yaml"
Type: String
Resources:
LambdaRoleForRuleExecution:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub ${AWS::StackName}-lambda-role
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action: 'sts:AssumeRole'
Principal:
Service: lambda.amazonaws.com
Policies:
- PolicyName: WriteCloudWatchLogs
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- 'logs:CreateLogGroup'
- 'logs:CreateLogStream'
- 'logs:PutLogEvents'
Resource: 'arn:aws:logs:*:*:*'
ApiGatewayStepFunctionsRole:
Type: AWS::IAM::Role
Properties:
Path: !Join ["", ["/", !Ref "AWS::StackName", "/"]]
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Sid: AllowApiGatewayServiceToAssumeRole
Effect: Allow
Action:
- 'sts:AssumeRole'
Principal:
Service:
- apigateway.amazonaws.com
Policies:
- PolicyName: CallStepFunctions
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- 'states:StartExecution'
Resource:
- !Ref Workflow
Start:
Type: AWS::Lambda::Function
Properties:
FunctionName: !Sub ${AWS::StackName}-start
Code: ../dist/src/step-functions
Handler: step-functions.start
Role: !GetAtt LambdaRoleForRuleExecution.Arn
Runtime: nodejs8.10
Timeout: 1
Wait3000:
Type: AWS::Lambda::Function
Properties:
FunctionName: !Sub ${AWS::StackName}-wait3000
Code: ../dist/src/step-functions
Handler: step-functions.wait3000
Role: !GetAtt LambdaRoleForRuleExecution.Arn
Runtime: nodejs8.10
Timeout: 4
Wait500:
Type: AWS::Lambda::Function
Properties:
FunctionName: !Sub ${AWS::StackName}-wait500
Code: ../dist/src/step-functions
Handler: step-functions.wait500
Role: !GetAtt LambdaRoleForRuleExecution.Arn
Runtime: nodejs8.10
Timeout: 2
End:
Type: AWS::Lambda::Function
Properties:
FunctionName: !Sub ${AWS::StackName}-end
Code: ../dist/src/step-functions
Handler: step-functions.end
Role: !GetAtt LambdaRoleForRuleExecution.Arn
Runtime: nodejs8.10
Timeout: 1
StateExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- !Sub states.${AWS::Region}.amazonaws.com
Action:
- 'sts:AssumeRole'
Policies:
- PolicyName: "StatesExecutionPolicy"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Action: "lambda:InvokeFunction"
Resource:
- !GetAtt Start.Arn
- !GetAtt Wait3000.Arn
- !GetAtt Wait500.Arn
- !GetAtt End.Arn
Workflow:
Type: AWS::StepFunctions::StateMachine
Properties:
StateMachineName: !Sub ${AWS::StackName}-state-machine
RoleArn: !GetAtt StateExecutionRole.Arn
DefinitionString: !Sub |
{
"Comment": "AWS Step Functions Example",
"StartAt": "Start",
"Version": "1.0",
"States": {
"Start": {
"Type": "Task",
"Resource": "${Start.Arn}",
"Next": "Parallel State"
},
"Parallel State": {
"Type": "Parallel",
"Next": "End",
"Branches": [
{
"StartAt": "Wait3000",
"States": {
"Wait3000": {
"Type": "Task",
"Resource": "${Wait3000.Arn}",
"End": true
}
}
},
{
"StartAt": "Wait500",
"States": {
"Wait500": {
"Type": "Task",
"Resource": "${Wait500.Arn}",
"End": true
}
}
}
]
},
"End": {
"Type": "Task",
"Resource": "${End.Arn}",
"End": true
}
}
}
RestApi:
Type: AWS::Serverless::Api
Properties:
StageName: !Ref Environment
Name: !Sub ${AWS::StackName}-api
DefinitionBody:
'Fn::Transform':
Name: AWS::Include
Parameters:
# s3 location of the swagger file
Location: !Ref SwaggerS3File
swagger.yaml
openapi: 3.0.0
info:
version: '1.0'
title: "pit-jv-lambda-examples"
description: POC API
license:
name: MIT
x-amazon-apigateway-request-validators:
Validate body:
validateRequestParameters: false
validateRequestBody: true
params:
validateRequestParameters: true
validateRequestBody: false
Validate body, query string parameters, and headers:
validateRequestParameters: true
validateRequestBody: true
paths:
/execute:
options:
x-amazon-apigateway-integration:
type: mock
requestTemplates:
application/json: |
{
"statusCode" : 200
}
responses:
"default":
statusCode: "200"
responseParameters:
method.response.header.Access-Control-Allow-Headers:
Fn::Sub: ${CorsHeaders}
method.response.header.Access-Control-Allow-Methods:
Fn::Sub: ${CorsMethods}
method.response.header.Access-Control-Allow-Origin:
Fn::Sub: ${CorsOrigin}
responseTemplates:
application/json: |
{}
responses:
200:
$ref: '#/components/responses/200Cors'
post:
x-amazon-apigateway-integration:
credentials:
Fn::GetAtt: [ ApiGatewayStepFunctionsRole, Arn ]
uri:
Fn::Sub: arn:aws:apigateway:${AWS::Region}:states:action/StartExecution
httpMethod: POST
type: aws
responses:
default:
statusCode: 200
responseParameters:
method.response.header.Access-Control-Allow-Headers:
Fn::Sub: ${CorsHeaders}
method.response.header.Access-Control-Allow-Origin:
Fn::Sub: ${CorsOrigin}
".*CREATION_FAILED.*":
statusCode: 403
responseParameters:
method.response.header.Access-Control-Allow-Headers:
Fn::Sub: ${CorsHeaders}
method.response.header.Access-Control-Allow-Origin:
Fn::Sub: ${CorsOrigin}
responseTemplates:
application/json: $input.path('$.errorMessage')
requestTemplates:
application/json:
Fn::Sub: |-
{
"input": "$util.escapeJavaScript($input.json('$'))",
"name": "$context.requestId",
"stateMachineArn": "${Workflow}"
}
summary: Start workflow
responses:
200:
$ref: '#/components/responses/200Empty'
403:
$ref: '#/components/responses/Error'
components:
schemas:
Error:
title: Error
type: object
properties:
code:
type: string
message:
type: string
responses:
200Empty:
description: Default OK response
200Cors:
description: Default response for CORS method
headers:
Access-Control-Allow-Headers:
schema:
type: "string"
Access-Control-Allow-Methods:
schema:
type: "string"
Access-Control-Allow-Origin:
schema:
type: "string"
Error:
description: Error Response
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
headers:
Access-Control-Allow-Headers:
schema:
type: "string"
Access-Control-Allow-Origin:
schema:
type: "string"
step-functions.js
exports.start = (event, context, callback) => {
console.log('start event', event);
console.log('start context', context);
callback(undefined, { function: 'start' });
};
exports.wait3000 = (event, context, callback) => {
console.log('wait3000 event', event);
console.log('wait3000 context', context);
setTimeout(() => {
callback(undefined, { function: 'wait3000' });
}, 3000);
};
exports.wait500 = (event, context, callback) => {
console.log('wait500 event', event);
console.log('wait500 context', context);
setTimeout(() => {
callback(undefined, { function: 'wait500' });
}, 500);
};
exports.end = (event, context, callback) => {
console.log('end event', event);
console.log('end context', context);
callback(undefined, { function: 'end' });
};