Invoking an AWS lambda function through step functions in Golang - amazon-web-services

I've got an AWS step function that invokes a lambda function written in Golang. For some reason it looks like the lambda function is not able to read the input to the step function.
The lambda function -
package main
import (
"fmt"
"github.com/aws/aws-lambda-go/lambda"
)
type InEvent struct {
Name string `json:"name"`
}
type OutEvent struct {
Greeting string `json:"greeting"`
}
func HandleRequest(name InEvent) (OutEvent, error) {
var result OutEvent
result.Greeting = fmt.Sprintf("Hello %s!", name.Name)
return result, nil
}
func main() {
lambda.Start(HandleRequest)
}
The step function -
{
"Comment": "A Hello World example of the Amazon States Language using Pass states",
"StartAt": "Invoke Lambda function",
"States": {
"Invoke Lambda function": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": "arn:aws:lambda:us-west-1:301438421794:function:SF1_1:$LATEST",
"Payload": {
"Input.$": "$"
}
},
"End": true
}
}
}
Input to the step function -
{
"name": "ABCDE"
}
Output of the step function -
{
"ExecutedVersion": "$LATEST",
"Payload": {
"greeting": "Hello !"
},
"SdkHttpMetadata": {
"AllHttpHeaders": {
"X-Amz-Executed-Version": [
"$LATEST"
],
"x-amzn-Remapped-Content-Length": [
"0"
],
"Connection": [
"keep-alive"
],
"x-amzn-RequestId": [
"5f297331-8b1a-49a0-9ad9-17a78ec1cfd0"
],
"Content-Length": [
"22"
],
"Date": [
"Fri, 25 Dec 2020 19:58:20 GMT"
],
"X-Amzn-Trace-Id": [
"root=1-5fe6445c-5cb6860d2559230940506a2f;sampled=0"
],
"Content-Type": [
"application/json"
]
},
"HttpHeaders": {
"Connection": "keep-alive",
"Content-Length": "22",
"Content-Type": "application/json",
"Date": "Fri, 25 Dec 2020 19:58:20 GMT",
"X-Amz-Executed-Version": "$LATEST",
"x-amzn-Remapped-Content-Length": "0",
"x-amzn-RequestId": "5f297331-8b1a-49a0-9ad9-17a78ec1cfd0",
"X-Amzn-Trace-Id": "root=1-5fe6445c-5cb6860d2559230940506a2f;sampled=0"
},
"HttpStatusCode": 200
},
"SdkResponseMetadata": {
"RequestId": "5f297331-8b1a-49a0-9ad9-17a78ec1cfd0"
},
"StatusCode": 200
}
I expect Output.Payload.greeting to be "Hello ABCDE!" but instead Output.Payload.greeting is "Hello !"
What went wrong here? Why is the name variable in the lambda function not storing the input correctly? Why does "ABCDE" turn into an empty string within the lambda function?
When I directly test the lambda function on the lambda console without using step functions, everything works fine.

The Payload part in your state machine definition is shaping the input passed to Lambda:
"Payload": {
"Input.$": "$"
}
So the actual input of Lambda will be:
{
Input:
{
name: 'ABCDE'
}
}
So you need to consider that in your Golang code or change that Payload part in your state machine definition.

Be careful when sharing the code and example where you have your account id as well being exposed
Below is the working state machine with your golang code
{
"Comment": "A Hello World example of the Amazon States Language using Pass states",
"StartAt": "Invoke Lambda function",
"States": {
"Invoke Lambda function": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": "arn:aws:lambda:us-east-1:301438421794:function:testfunction",
"Payload": {
"name.$": "$.name"
}
},
"ResultPath": "$.lambdaOutput",
"End": true
}
}
}
Produces the output like below:
{
"name": "suresh",
"lambdaOutput": {
"ExecutedVersion": "$LATEST",
"Payload": {
"greeting": "Hello suresh!"
},
"SdkHttpMetadata": {
....
}
You can further slice the output by amending like this
"OutputPath": "$.lambdaOutput.Payload",
This will give you exact output
{
"greeting": "Hello suresh!"
}
Take a look here. This is the best resource I can recommend to get started with step function.
For handling the data in between functions keep this handy. For handling the inputs and outputs this also is pretty useful.

Related

Cannot add variable from context on QueryParameters on Step Function API Gateway call

I'm trying two consecutive API Gateway calls from a Step Function, using values from the first result on the second. I tried already on a POST call and it worked, but now I can't do it on a GET call. This is my step function:
{
"Comment": "A Hello World example of the Amazon States Language using Pass states",
"StartAt": "Api Call",
"States": {
"Api Call": {
"Type": "Task",
"Resource": "arn:aws:states:::apigateway:invoke",
"Parameters": {
"ApiEndpoint": "***************.us-east-1.amazonaws.com",
"Method": "GET",
"Path": "universities",
"RequestBody": {},
"AuthType": "NO_AUTH"
},
"ResultSelector": {
"logWord.$": "$.StatusText"
},
"Next": "Api Call 2"
},
"Api Call 2": {
"Type": "Task",
"Resource": "arn:aws:states:::apigateway:invoke",
"Parameters": {
"ApiEndpoint": "***************.us-east-1.amazonaws.com",
"Method": "GET",
"Path": "logging",
"Headers": {
"Content-Type": ["application/json"]
},
"QueryParameters": {
"logWord.$": "$.logWord"
},
"AuthType": "NO_AUTH"
},
"End": true
}
}
}
The error that I get is the following:
{
"error": "States.Runtime",
"cause": "An error occurred while executing the state 'Api Call 2' (entered at the event id #7). The Parameters '{\"ApiEndpoint\":\"***************.us-east-1.amazonaws.com\",\"Method\":\"GET\",\"Path\":\"logging\",\"Headers\":{\"Content-Type\":[\"application/json\"]},\"QueryParameters\":{\"logWord\":\"OK\"},\"AuthType\":\"NO_AUTH\"}' could not be used to start the Task: [The value of the field 'QueryParameters' has an invalid format]"
}
According to the documentation, query parameters should be between brackets, but if I'm using a context variable I can't put the brackets. The console doesn't allow me to even save (The value for the field 'logWord.$' must be a STRING that contains a JSONPath but was an ARRAY).
If anyone knows how to make GET call using context variables would be greatly appreciated.
Looks like QueryParameters wants a list of strings. Use the States.Array intrinsic function to convert your interpolated string to a [string] list.
"QueryParameters": {
"logWord.$": "States.Array($.logWord)" // ["OK"]
},

State Machine Will Not Accept Input Path

I'm sure someone will point me to an immediate solution, but I've been at this for hours, so I'm just going to ask.
I cannot get a State Machine to accept an initial input. The intent is to set up an EventBridge trigger pointed at the State Machine with a static JSON passed to the SM to initiate with the proper parameters. In development, I'm just using Step Functions option to pass a JSON as the initial input when you select "New Execution".
This is the input:
{"event":{
"country": "countryA",
"landing_bucket": "aws-glue-countryA-inputs",
"landing_key": "countryA-Bucket/prefix/filename.csv",
"forecast_bucket": "aws-forecast-countryA",
"forecast_key": "inputs/",
"date_start": "2018-01-01",
"validation": "False",
"validation_size": 90
}
}
When looking at what is passed at the ExecutionStarted log entry:
{
"input": {
"country": "countryA",
"landing_bucket": "aws-glue-countryA-inputs",
"landing_key": "countryA-Bucket/prefix/filename.csv",
"forecast_bucket": "aws-forecast-countryA",
"forecast_key": "inputs/",
"date_start": "2018-01-01",
"validation": "False",
"validation_size": 90
}
,
"inputDetails": {
"truncated": false
},
"roleArn": "arn:aws:iam::a-valid-service-role"
}
This is the State Machine:
"Comment": "A pipeline!",
"StartAt": "Invoke Preprocessor",
"States": {
"Invoke Preprocessor": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"InputPath": "$.input",
"Parameters": {
"FunctionName": "arn:aws:lambda:my-lambda-arn:$LATEST"
},
"Next": "EndSM"
},
"EndSM": {
"Type": "Pass",
"Result": "Ended",
"End": true
}
}
}
I've tried nearly anything I can think of from changing the InputPath to assigning the "input" dictionary directly to a variable:
"$.event":"$.input"
To drilling down to the individual variables and assigning those directly like:
"$.country:"$.country". I've also used the new Step Functions Data Flow Simulator and can't get anywhere. If anyone has any thoughts, I'd really appreciate it.
Thanks!
Edited for correct solution:
You need to set the Payload.$ parameter to $. That will pass in the entire input object to the lambda.
{
"Comment": "A pipeline!",
"StartAt": "Invoke Preprocessor",
"States": {
"Invoke Preprocessor": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": "arn:aws:lambda:my-lambda-arn:$LATEST",
"Payload.$": "$"
},
"Next": "EndSM"
},
"EndSM": {
"Type": "Pass",
"Result": "Ended",
"End": true
}
}
}
Another thing you could do is specify the input in the parameters, this will allow you to specify only all/certain parts of the json to pass in.
{
"Comment": "A pipeline!",
"StartAt": "Invoke Preprocessor",
"States": {
"Invoke Preprocessor": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"InputPath": "$",
"Parameters": {
"FunctionName": "arn:aws:lambda:my-lambda-arn:$LATEST",
"input_event.$": "$.event"
},
"Next": "EndSM"
},
"EndSM": {
"Type": "Pass",
"Result": "Ended",
"End": true
}
}
}
From the code perspective you could just reference it like so (python):
input = event['input_event']

Error when invoking lambda from StepFunction with paypload

I have a lambda and it's one of the steps of a StepFunction statemachine, the lambda handler looks like this below :
def lambda_handler(event, context):
year = event['year'] #payload
month = event['month'] #payload
example_function(year, month)
This lambda execution succeeded if f I start a run in Lambda itself with payload:
{
"year": "2019",
"month": "06"
}
However if I kick off a stepfunction run, it will fail with error:
[ERROR] KeyError: 'year'
Traceback (most recent call last):
File "/var/task/xxx.py", line 34, in lambda_handler
year = event['year']
My stepfunction definition is:
{
"Comment": "xxxxxxxxx",
"StartAt": "invoke lambda",
"States": {
"invoke lambda" : {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName":"xxxxxxxxxx",
"Payload": {
"Input": {
"year": "2019",
"month": "06"
}
}
}
"End": true
}
}
}
I'm not sure how to debug and solve this issue, might someone be able to help please?
There are two ways to call the lambda function from step function.
First Method, directly by giving the Resource name as lambda Arn and passing the entire input to Parameters. This passes given input as is to Lambda
{
"StartAt":"invoke-lambda",
"States":{
"invoke-lambda":{
"End":true,
"Type":"Task",
"Resource":"arn:aws:lambda:us-east-1:660008888333:function:HelloWorld",
"Parameters":{
"year":"2019",
"month":"06"
}
}
}
}
Second Method using Resource lambda:invoke and Payload with Json body(no need to wrap within Input)
{
"StartAt":"invoke-lambda",
"States":{
"invoke-lambda":{
"End":true,
"Type":"Task",
"Resource":"arn:aws:states:::lambda:invoke",
"Parameters":{
"FunctionName":"arn:aws:lambda:us-east-1:660008888333:function:HelloWorld",
"Payload":{
"year":"2019",
"month":"06"
}
}
}
}
}
If input to Lambda is from either previous step or from input to step function input, we can use "Payload.$": "$" or even "Payload.$": "$.subPath"(to pass part of json)
Also it is recommended to add a Retry for exceptions that are triggered because of Exceptions caused by AWS.
{
"StartAt": "invoke-lambda",
"States": {
"invoke-lambda": {
"End": true,
"Retry": [
{
"ErrorEquals": [
"Lambda.ServiceException",
"Lambda.AWSLambdaException",
"Lambda.SdkClientException"
],
"IntervalSeconds": 2,
"MaxAttempts": 6,
"BackoffRate": 2
}
],
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": "arn:aws:lambda:us-east-1:660008888333:function:HelloWorld",
"Payload.$": "$",
"InvocationType": "RequestResponse"
}
}
}
}

aws step functions pass data from lambda to lambda

I've been researching on how to pass data from a Lambda to another Lambda in a Step Function and this is what i got.
I have this dummy lambda that pass the data name:
exports.lambdaHandler = async (event, context, callback) => {
const name = 'test';
callback(null, { name });
}
to another lambda, where i try to get the data in this way, but is not working:
const name = event.name; //this returns undefined
Based on this tutorial, this should be enough but it doesn't work. Can you point me in what direction should i go? Do i have to use the InputPath, ResultPath properties of the states machines?
[Update]
This is the State machine definition:
{
"Comment": "commen test",
"StartAt": "FunctionOne",
"States": {
"FunctionOne": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": <arn FunctionOne>
},
"Next": "FunctionTwo"
},
"FunctionTwo": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": <arn FunctionTwo>
},
"End": true
}
}
}
Try this
{
"Comment": "commen test",
"StartAt": "FunctionOne",
"States": {
"FunctionOne": {
"Type": "Task",
"Resource": "<arn FunctionOne>",
"Next": "FunctionTwo"
},
"FunctionTwo": {
"Type": "Task",
"Resource": "<arn FunctionTwo>",
"End": true
}
}
}

Unable to execute Lambda in Async mode via API Gateway POST request

Why currently there no way to execute AWS Lambda in asynchronous mode via Gateway API without involving intermediary Lambda just for calling invoke() method?
Even if i add integration like this:
r = client.put_integration(
restApiId=rest_api_id,
resourceId=resource_id,
httpMethod='POST',
type='AWS',
integrationHttpMethod='POST',
uri=uri,
requestParameters={
'integration.request.header.X-Amz-Invocation-Type': "'Event'",
'integration.request.header.Invocation-Type': "'Event'"
}
)
It still executed synchronously...
Are there some platform limitation or so?
I have an example Swagger document which you can switch invocation type to Lambda function. I guess you already got how to map the header to trigger the different invocation types, but I think you might forget to deploy the API.
Swagger
{
"swagger": "2.0",
"info": {
"version": "2016-02-11T22:00:31Z",
"title": "LambdaAsync"
},
"host": "<placeholder>",
"basePath": "<placeholder>",
"schemes": [
"https"
],
"paths": {
"/": {
"get": {
"produces": [
"application/json"
],
"parameters": [
{
"name": "X-Amz-Invocation-Type",
"in": "header",
"required": false,
"type": "string"
}
],
"responses": {
"200": {
"description": "200 response",
"schema": {
"$ref": "#/definitions/Empty"
}
}
},
"x-amazon-apigateway-integration": {
"passthroughBehavior": "when_no_match",
"httpMethod": "POST",
"uri": "arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-1:<account>:function:<function_name>/invocations?Qualifier=$LATEST",
"responses": {
"default": {
"statusCode": "200"
}
},
"requestParameters": {
"integration.request.header.X-Amz-Invocation-Type": "method.request.header.X-Amz-Invocation-Type"
},
"type": "aws"
}
}
}
},
"definitions": {
"Empty": {
"type": "object",
"title": "Empty Schema"
}
}
}