AWS CDK Step Functions - Response Always Null - amazon-web-services

I'm using AWS CDK Step Function constructs to create a simple workflow. I can invoke the first Lambda fine and that subsequently invokes next Lambda. However, on the second Lambda my input is fine as expected but the output from lambda task always returns Payload: null as response. I do not intend this behaviour and want to return data inside the Lambda via output key to be passed to next task.
export const bulkSummaryHandler = (event) => {
try {
console.log('LAMBA SUMMARY!', event);
return { output: { status: 'finished' } };
} catch (error) {
return handleError(error);
}
};
My CDK code
const getUserCsvFileTask = new tasks.LambdaInvoke(ctx.stack, 'getUserCsvFileTask', {
lambdaFunction: getUserCsvFileFn,
comment: 'fetch user uploaded csv from csv-integration-service',
inputPath: '$',
resultPath: '$.taskResult',
outputPath: '$.taskResult.Payload'
});
const bulkSummaryTask = new tasks.LambdaInvoke(ctx.stack, 'bulkProcessingSummaryTask', {
lambdaFunction: bulkSummaryFn,
comment: 'summarise bulk processing',
inputPath: '$'
});
const definition = stepfunctions.Chain.start(getUserCsvFileTask).next(bulkSummaryTask).next(nextLambdaTask);
The response I get from second Lambda 'bulk summary task' invoked in Payload Key is always null. It's not clear to me why I am getting null and I'm out of ideas as to why. Any ideas would be great help.
{
"ExecutedVersion": "$LATEST",
"Payload": null,
"SdkHttpMetadata": {
"AllHttpHeaders": {
"X-Amz-Executed-Version": [
"$LATEST"
],
"x-amzn-Remapped-Content-Length": [
"0"
],
"Connection": [
"keep-alive"
],
"x-amzn-RequestId": [
"fed8b1bd-d188-4425-ade7-ce2723aef4c8"
],
"Content-Length": [
"4"
],
"Date": [
"Wed, 21 Sep 2022 22:54:00 GMT"
],
"X-Amzn-Trace-Id": [
"root=1-632b9607-0e451e4c5dd4c21c7a3eaa8b;sampled=1"
],
"Content-Type": [
"application/json"
]
},
"HttpHeaders": {
"Connection": "keep-alive",
"Content-Length": "4",
"Content-Type": "application/json",
"Date": "Wed, 21 Sep 2022 22:54:00 GMT",
"X-Amz-Executed-Version": "$LATEST",
"x-amzn-Remapped-Content-Length": "0",
"x-amzn-RequestId": "fed8b1bd-d188-4425-ade7-ce2723aef4c8",
"X-Amzn-Trace-Id": "root=1-632b9607-0e451e4c5dd4c21c7a3eaa8b;sampled=1"
},
"HttpStatusCode": 200
},
"SdkResponseMetadata": {
"RequestId": "fed8b1bd-d188-4425-ade7-ce2723aef4c8"
},
"StatusCode": 200
}

Ah I've been incredibly stupid. The handler needs to be async there's no callback, it won't return anything.
export const bulkSummaryHandler = async (event) => {
try {
console.log('LAMBA SUMMARY!', event);
return { output: { status: 'finished' } };
} catch (error) {
return handleError(error);
}
};

Related

How to enable access in Google Cloud Storage to `x-goog-hash` header from client side (browser) [duplicate]

I'm trying to access the md5 that returns from uploading file to Google Cloud Storage.
But, I can only get the other headers, not the x-goog-hash that contains the md5.
async uploadVideo(uploadURL: string, file: any) {
return axios
.put(uploadURL, file, {
headers: {
"content-type": "application/octet-stream",
},
})
.then((response) => console.log(response))
.catch((error) => {
console.log("error: " + error);
});
}
This is what the console.log shows:
{
"data": "",
"status": 200,
"statusText": "",
"headers": {
"access-control-allow-origin": "*",
"content-length": "0",
"content-type": "text/html; charset=UTF-8",
"date": "Tue, 01 Mar 2022 09:22:11 GMT",
"server": "UploadServer",
"x-guploader-uploadid": "ADPycdvmU7_-9wqmePl5m_jzzdCWmr84AZwEbv2SJJ99kJLkq2eYzpQqrMaK4mHu_RiNVI3WzTkVH5Y"
},
"config": {
"transitional": {
"silentJSONParsing": true,
"forcedJSONParsing": true,
"clarifyTimeoutError": false
},
"transformRequest": [
null
],
"transformResponse": [
null
],
"timeout": 0,
"xsrfCookieName": "XSRF-TOKEN",
"xsrfHeaderName": "X-XSRF-TOKEN",
"maxContentLength": -1,
"maxBodyLength": -1,
"headers": {
"Accept": "application/json, text/plain, */*",
"Content-Type": "application/octet-stream"
},
"method": "put",
"url": "https://storage.googleapis.com/.........
"data": {}
},
"request": {}
}

Invoking an AWS lambda function through step functions in Golang

I've got an AWS step function that invokes a lambda function written in Golang. For some reason it looks like the lambda function is not able to read the input to the step function.
The lambda function -
package main
import (
"fmt"
"github.com/aws/aws-lambda-go/lambda"
)
type InEvent struct {
Name string `json:"name"`
}
type OutEvent struct {
Greeting string `json:"greeting"`
}
func HandleRequest(name InEvent) (OutEvent, error) {
var result OutEvent
result.Greeting = fmt.Sprintf("Hello %s!", name.Name)
return result, nil
}
func main() {
lambda.Start(HandleRequest)
}
The step function -
{
"Comment": "A Hello World example of the Amazon States Language using Pass states",
"StartAt": "Invoke Lambda function",
"States": {
"Invoke Lambda function": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": "arn:aws:lambda:us-west-1:301438421794:function:SF1_1:$LATEST",
"Payload": {
"Input.$": "$"
}
},
"End": true
}
}
}
Input to the step function -
{
"name": "ABCDE"
}
Output of the step function -
{
"ExecutedVersion": "$LATEST",
"Payload": {
"greeting": "Hello !"
},
"SdkHttpMetadata": {
"AllHttpHeaders": {
"X-Amz-Executed-Version": [
"$LATEST"
],
"x-amzn-Remapped-Content-Length": [
"0"
],
"Connection": [
"keep-alive"
],
"x-amzn-RequestId": [
"5f297331-8b1a-49a0-9ad9-17a78ec1cfd0"
],
"Content-Length": [
"22"
],
"Date": [
"Fri, 25 Dec 2020 19:58:20 GMT"
],
"X-Amzn-Trace-Id": [
"root=1-5fe6445c-5cb6860d2559230940506a2f;sampled=0"
],
"Content-Type": [
"application/json"
]
},
"HttpHeaders": {
"Connection": "keep-alive",
"Content-Length": "22",
"Content-Type": "application/json",
"Date": "Fri, 25 Dec 2020 19:58:20 GMT",
"X-Amz-Executed-Version": "$LATEST",
"x-amzn-Remapped-Content-Length": "0",
"x-amzn-RequestId": "5f297331-8b1a-49a0-9ad9-17a78ec1cfd0",
"X-Amzn-Trace-Id": "root=1-5fe6445c-5cb6860d2559230940506a2f;sampled=0"
},
"HttpStatusCode": 200
},
"SdkResponseMetadata": {
"RequestId": "5f297331-8b1a-49a0-9ad9-17a78ec1cfd0"
},
"StatusCode": 200
}
I expect Output.Payload.greeting to be "Hello ABCDE!" but instead Output.Payload.greeting is "Hello !"
What went wrong here? Why is the name variable in the lambda function not storing the input correctly? Why does "ABCDE" turn into an empty string within the lambda function?
When I directly test the lambda function on the lambda console without using step functions, everything works fine.
The Payload part in your state machine definition is shaping the input passed to Lambda:
"Payload": {
"Input.$": "$"
}
So the actual input of Lambda will be:
{
Input:
{
name: 'ABCDE'
}
}
So you need to consider that in your Golang code or change that Payload part in your state machine definition.
Be careful when sharing the code and example where you have your account id as well being exposed
Below is the working state machine with your golang code
{
"Comment": "A Hello World example of the Amazon States Language using Pass states",
"StartAt": "Invoke Lambda function",
"States": {
"Invoke Lambda function": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": "arn:aws:lambda:us-east-1:301438421794:function:testfunction",
"Payload": {
"name.$": "$.name"
}
},
"ResultPath": "$.lambdaOutput",
"End": true
}
}
}
Produces the output like below:
{
"name": "suresh",
"lambdaOutput": {
"ExecutedVersion": "$LATEST",
"Payload": {
"greeting": "Hello suresh!"
},
"SdkHttpMetadata": {
....
}
You can further slice the output by amending like this
"OutputPath": "$.lambdaOutput.Payload",
This will give you exact output
{
"greeting": "Hello suresh!"
}
Take a look here. This is the best resource I can recommend to get started with step function.
For handling the data in between functions keep this handy. For handling the inputs and outputs this also is pretty useful.

AWS - How to get Cloudfront metrics using Javascript SDK

I am trying to get Cloudfront metrics using the JS SDK for AWS but I am not getting anything back.
I am not sure what I am doing wrong but I have isolated this NOT to be with:
Permissions (gave it a full admin account for testing purposes)
Region. North Virginia (for CloudFront)
Basic params like: StartDate, EndDate, DistributionID
My code is as below (simplified):
var AWS = require('aws-sdk');
AWS.config.update({
accessKeyId: "accessKeyId",
secretAccessKey: "secretAccessKey",
apiVersion: '2017-10-25',
region: 'us-east-1'
});
var cloudwatchmetrics = new AWS.CloudWatch();
var cloudFrontParams = {
"StartTime": 1518867432,
"EndTime": 1518868032,
"MetricDataQueries": [
{
"Id": "m1",
"MetricStat": {
"Metric": {
"Dimensions": [
{
"Name": "DistributionId",
"Value": "ValueOfDistribution"
},
{
"Name": "Region",
"Value": "Global"
}
],
"MetricName": "Requests",
"Namespace": "AWS/CloudFront"
},
"Stat": "Sum",
"Period": 3600
},
"ReturnData": true
}
]
};
cloudwatchmetrics.getMetricData(cloudFrontParams, function (err, data) {
if (err) {
console.log(err);
}else{
console.log(JSON.stringify(data));
}
});
This is what I get back (it's not erroring out):
{
"ResponseMetadata":{
"RequestId":"xxxxxxxxxxxxxxxxxxxxx"
},
"MetricDataResults":[
{
"Id":"m1",
"Label":"Requests",
"Timestamps":[
],
"Values":[
],
"StatusCode":"Complete",
"Messages":[
]
}
]
}
The issue was with the StartTime, it was too far back in time. What I have in the post translates to: Saturday, February 17, 2018
Hopefully this helps someone someday.

How to test a GET and POST call from inside of Lambda

I am learning Lambda, have created a Cloud Watch Metrics which triggers Lambda every one minute. At present, my Lambda function is executing properly. But I want to make a fake GET and POST call from inside of Lambda. Something like below:
var https = require('https');
exports.handler = (event, context, callback) => {
var params = {
host: "example.com",
path: "/api/v1/yourmethod"
};
var req = https.request(params, function(res) {
let data = '';
console.log('STATUS: ' + res.statusCode);
res.setEncoding('utf8');
res.on('data', function(chunk) {
data += chunk;
});
res.on('end', function() {
console.log("DONE");
console.log(JSON.parse(data));
});
});
req.end();
};
Question - Since I don't have any GET and POST endpoint/Code hosted in AWS is there any way I can test whether my GET call and POST call from inside of Lambda is working fine? Is there any way in AWS to put together a fake endpoint or something in AWS and make REST call to it for testing purposes?
You can use JSONPlaceholder's APIs for testing calls to external APIs,
They have released Fake Online REST APIs for Testing and Prototyping
You can test calling different http method calls like GET, PUT, POST, DELETE
https://jsonplaceholder.typicode.com/
In case you are using Lambda proxy integration Event will contents a lot of additional information about request: https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html
{
"message": "Hello me!",
"input": {
"resource": "/{proxy+}",
"path": "/hello/world",
"httpMethod": "POST",
"headers": {
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate",
"cache-control": "no-cache",
"CloudFront-Forwarded-Proto": "https",
"CloudFront-Is-Desktop-Viewer": "true",
"CloudFront-Is-Mobile-Viewer": "false",
"CloudFront-Is-SmartTV-Viewer": "false",
"CloudFront-Is-Tablet-Viewer": "false",
"CloudFront-Viewer-Country": "US",
"Content-Type": "application/json",
"headerName": "headerValue",
"Host": "gy415nuibc.execute-api.us-east-1.amazonaws.com",
"Postman-Token": "9f583ef0-ed83-4a38-aef3-eb9ce3f7a57f",
"User-Agent": "PostmanRuntime/2.4.5",
"Via": "1.1 d98420743a69852491bbdea73f7680bd.cloudfront.net (CloudFront)",
"X-Amz-Cf-Id": "pn-PWIJc6thYnZm5P0NMgOUglL1DYtl0gdeJky8tqsg8iS_sgsKD1A==",
"X-Forwarded-For": "54.240.196.186, 54.182.214.83",
"X-Forwarded-Port": "443",
"X-Forwarded-Proto": "https"
},
"multiValueHeaders":{
'Accept':[
"*/*"
],
'Accept-Encoding':[
"gzip, deflate"
],
'cache-control':[
"no-cache"
],
'CloudFront-Forwarded-Proto':[
"https"
],
'CloudFront-Is-Desktop-Viewer':[
"true"
],
'CloudFront-Is-Mobile-Viewer':[
"false"
],
'CloudFront-Is-SmartTV-Viewer':[
"false"
],
'CloudFront-Is-Tablet-Viewer':[
"false"
],
'CloudFront-Viewer-Country':[
"US"
],
'':[
""
],
'Content-Type':[
"application/json"
],
'headerName':[
"headerValue"
],
'Host':[
"gy415nuibc.execute-api.us-east-1.amazonaws.com"
],
'Postman-Token':[
"9f583ef0-ed83-4a38-aef3-eb9ce3f7a57f"
],
'User-Agent':[
"PostmanRuntime/2.4.5"
],
'Via':[
"1.1 d98420743a69852491bbdea73f7680bd.cloudfront.net (CloudFront)"
],
'X-Amz-Cf-Id':[
"pn-PWIJc6thYnZm5P0NMgOUglL1DYtl0gdeJky8tqsg8iS_sgsKD1A=="
],
'X-Forwarded-For':[
"54.240.196.186, 54.182.214.83"
],
'X-Forwarded-Port':[
"443"
],
'X-Forwarded-Proto':[
"https"
]
},
"queryStringParameters": {
"name": "me",
"multivalueName": "me"
},
"multiValueQueryStringParameters":{
"name":[
"me"
],
"multivalueName":[
"you",
"me"
]
},
"pathParameters": {
"proxy": "hello/world"
},
"stageVariables": {
"stageVariableName": "stageVariableValue"
},
"requestContext": {
"accountId": "12345678912",
"resourceId": "roq9wj",
"stage": "testStage",
"requestId": "deef4878-7910-11e6-8f14-25afc3e9ae33",
"identity": {
"cognitoIdentityPoolId": null,
"accountId": null,
"cognitoIdentityId": null,
"caller": null,
"apiKey": null,
"sourceIp": "192.168.196.186",
"cognitoAuthenticationType": null,
"cognitoAuthenticationProvider": null,
"userArn": null,
"userAgent": "PostmanRuntime/2.4.5",
"user": null
},
"resourcePath": "/{proxy+}",
"httpMethod": "POST",
"apiId": "gy415nuibc"
},
"body": "{\r\n\t\"a\": 1\r\n}",
"isBase64Encoded": false
}
}

CloudFormation Custom Resource responseKey

I have got lambda backed Custom Stack in CloudFormation , So I need the fetch function output and put it to the AWS Console, how I can handle this problem?
My Stack is shown as below ;
"CreateExistingVPC": {
"Type": "Custom::CreateExistingVPC",
"Properties": {
"ServiceToken": { "Fn::If": ["LambdaAvailable",{ "Fn::GetAtt": [ "CustomLogic", "Outputs.LambdaAttachHostedZoneArn" ] }, { "Ref": "AWS::NoValue" } ] },
"Region": { "Ref": "AWS::Region" },
"HostedZoneId": { "Ref": "InternalHostedZone" },
"VpcId": { "Fn::GetAtt": [ "VPC", "Outputs.VPC" ] }
}
}
},
"Outputs": {
"Route53VPC": {
"Description": "ExistingRoute53VPCStatus",
"Value": { "Fn::GetAtt": [ "CreateExistingVPC", "{ ??????? }" ] }
}
}
In actually, I have found some answers but 'response key' not worked in my case , how I can find response key ??
AWS Cloudformation, Output value from Custom Resource
You need to use the variable you are using to return your response. e.g. (nodeJs)
module.exports.createPoolList = (event, context, callback) => {
if (event.RequestType == 'Create') {
let array = event.ResourceProperties.OpsPoolArnList.split(",");
array.push(event.ResourceProperties.UserPool);
let response = {
'list': array.join(),
};
sendresponse(event, "SUCCESS", response, "");
}
if (event.RequestType == 'Delete') {
sendresponse(event, "SUCCESS", null, "");
}
callback(null, "");
};
Here list is the variable which contains my output & returning in my response. The built payload looks like
let payload = {
'StackId': event.StackId,
'Status' : responsestatus,
'Reason' : reason,
'RequestId': event.RequestId,
'LogicalResourceId': event.LogicalResourceId,
'PhysicalResourceId': event.LogicalResourceId + 'qwerty',
'Data': response
};
And I refer to this in my script as
!GetAtt <ResourceName>.list
Hope it helps.