When using a Lambda Function to trigger a Step Function, how do you get the output of the function that triggered it?
Ok, so if you want to pass an input to a Step Function execution (or, more exactly, your 'State Machine''s execution), you just need to set said input the input property when calling StartExecution (see AWS Documentation: Start Execution)
In your case, it would most likely be your lambda's last step before calling it's callback.
If it's a node js lambda, that's what it would look like
const AWS = require("aws-sdk");
const stepfunctions = new AWS.StepFunctions();
exports.myHandler = function(event, context, callback) {
... your function's code
const params = {
stateMachineArn: 'YOUR_STATE_MACHINE_ARN', /* required */
input: 'STRINGIFIED INPUT',
name: 'AN EXECUTION NAME (such as an uuid or whatever)'
};
stepfunctions.startExecution(params, function(err, data) {
if (err) callback(err); // an error occurred
else callback(null, "some success message"); // successful response
});
}
Alternatively, if your payload is too big, you could store the data in S3 or DynamoDB and pass the reference to it as your State Machine's execution's input.
Related
I have a lambda functions in one AZ in EU and another one in us-east-1 to be used via CloudFront triggers.
CloudFront --> #edge function -> lambda function
Sometimes, it takes a while for the second lambda invoke to be finished which hits the lambda#edge limits. It would be fine if it happens in async but I don't see any results when I run it async. Here is the code:
"use strict";
const AWS = require("aws-sdk");
AWS.config.update({
region: "eu-west-1",
});
const querystring = require("querystring");
exports.handler = async (event, context) => {
let request = event.Records[0].cf.request;
let params = querystring.parse(request.querystring);
if (params.key) {
const payload = {
/* my payload */
};
const lambda_params = {
FunctionName: "lambda-func-name",
Payload: JSON.stringify(payload),
};
const lambda = new AWS.Lambda();
const resp= await lambda.invoke(lambda_params);
console.log("Finished");
} else {
// allow the response to pass through
return {
"status":404,
"body":"an error"
}
}
};
the second lambda func would process some images and putting the results in the S3, but when I call it async, I don't see any results. Am I missing something?
First of all, a lengthy runtime seems like a very bad use case for Lambda#Edge. Maybe you should redesign your system to handle that.
Please note that Lambda#Edge has a function timeout limit of maximum 30 seconds in case of origin events and only 5 seconds for viewer events. Moreover, there is a compute utilization limit:
CloudFront Functions have a limit on the time they can take to run,
measured as compute utilization. Compute utilization is a number
between 0 and 100 that indicates the amount of time that the function took to run as a percentage of the maximum allowed time. For example, a compute utilization of 35 means that the function completed in 35% of the maximum allowed time.
With respect to your current design, in order to invoke Lambda asynchronously the invoke function should specify the correct invocation type argument. So instead
const lambda_params = {
FunctionName: "lambda-func-name",
Payload: JSON.stringify(payload),
};
you should use
const lambda_params = {
FunctionName: "lambda-func-name",
InvocationType: "Event", // <= you are missing this parameter
Payload: JSON.stringify(payload),
};
By default, the InvocationType value is set to RequestResponse. This means synchronous invocation. See docs for reference.
I have an external service which is called by a state in aws step function, now from this external service i want to send "sendTaskSuccess" to the particular aws step machine. How can i do that. I know that we can call "sendSuccessToken" from lambda but want to know how to use "sendSuccessToken" without lambda.
The state machine ARN is not used .The step in the state machine that await a callback have a taskToken which uniquely identifies a specific task and a specific execution of state machine.
The ideal way to do this would be :
Step 1:
Store the taskToken in a persistent storage before invoking the external service
export const lambdaHandler: ProxyHandler = async (event, context) => {
const evt = JSON.parse(JSON.stringify(event));
const taskToken = evt['token'];
const dbObject =
{
SomeHelpfulInfo: 'set a suitable identifier',
taskToken:taskToken
};
StoreTaskToken(dbObject);
//call external service as per your logic
//
}
Step 2:
Read the taskToken from the persistent storage and invoke the step function task success
import { StepFunctions } from 'aws-sdk';
const stepfunctions = new StepFunctions();
const taskToken = GetTaskTokenFromDb(); //Have your implementation to get taskToken
const params = {
output: '',
taskToken: taskToken
};
let result = await stepfunctions.sendTaskSuccess(params).promise();
console.log('State machine - callback completed..');
I currently have an api called “cms-api”, which contains a dynamodb scan function in it.
getOrganization.js
'use strict'
const AWS = require('aws-sdk');
exports.handler = async function (event, context, callback) {
const documentClient = new AWS.DynamoDB.DocumentClient();
let responseBody = "";
const params = {
TableName : "Organization"
};
try{
const data = await documentClient.scan(params).promise();
}catch(err){
responseBody = `Unable to get Organization: ${err}`;
}
}
“Organization” Table has below attributes
------------------------------------------
Id isActive name
1 true tim
2 false tom
3 true ken
4 true joe
------------------------------------------
Later, I create another api in api gateway called web-api,
I want to use the same lambda function getOrganization.js in my resource.
But the getOrganization.js should only return the data with isActive = true.
Is it possible? Or should I create a new lambda function every time?
Attached my consideration before.
should I create Public api based on the current internal api
Generally, you would consider using stage variables in your API gateway. They allow you:
You can also use stage variables to pass configuration parameters to a Lambda function through your mapping templates. For example, you might want to reuse the same Lambda function for multiple stages in your API, but the function should read data from a different Amazon DynamoDB table depending on which stage is being called. In the mapping templates that generate the request for the Lambda function, you can use stage variables to pass the table name to Lambda.
I want to run some code only on the first execution of a Lambda version. (NB: I'm not referring to a cold start scenario as this will occur more than once).
computeInstanceInvocationCount is unfortunately reset to 0 upon every cold start.
functionVersion is an available property but unless I store this in memory outside the lambda I cannot calculate if it is indeed the first execution.
Is it possible to deduce this based on runtime values in event or context? Or is there any other way?
There is no way of knowing if this is the first time that a Lambda has ever run from any information passed into the Lambda.
You would have to include functionality to check elsewhere by setting a flag or parameter there, remember though that multiple copies of the Lambda could be invoked at the same time so any data store for this would presumably need to be transactional to ensure that it occurs only once.
One way that you can try is to use AWS parameter store.
On every deployment update the parameter store value with
{"version":"latest","is_firsttime":true}
So run the below command after deployment
aws secretsmanager update-secret --secret-id demo --secret-string '{"version":"latest","is_firsttime":true}'
So this is something that we need to make sure before deployment.
Now we can set logic inside the lambda, in the demo we will look into is_firsttime only.
var AWS = require('aws-sdk'),
region = "us-west-2",
secretName = "demo",
secret,
decodedBinarySecret;
var client = new AWS.SecretsManager({
region: region
});
client.getSecretValue({SecretId: secretName}, function(err, data) {
secret = data.SecretString;
secret=JSON.parse(secret)
if ( secret.is_firsttime == true)
{
console.log("lambda is running first time")
// any init operation here
// init completed, now we are good to set back it `is_firsttime` to false
var params = {
Description: "Init completeed, updating value at date or anythign",
SecretId: "demo",
SecretString : '[{ "version" : "latest", "is_firsttime": false}]'
};
client.updateSecret(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data); // successful response
});
}
else{
console.log("init already completed");
// rest of logic incase of not first time
}
})
This is just a demo code that will work in a non-lambda environment, adjust it accordingly.
Expected response for first time
{
ARN: 'arn:aws:secretsmanager:us-west-2:12345:secret:demo-0Nlyli',
Name: 'demo',
VersionId: '3ae6623a-1111-4a41-88e5-12345'
}
Second time
init already completed
I'm trying to use RDSDataService to query an Aurora Serverless database. When I'm trying to query, my lambda just times out (I've set it up to 5 minutes just to make sure it isn't a problem with that). I have 1 record in my database and when I try to query it, I get no results, and neither the error or data flows are called. I've verified executeSql is called by removing the dbClusterOrInstanceArn from my params and it throw the exception for not having it.
I have also run SHOW FULL PROCESSLIST in the query editor to see if the queries were still running and they are not. I've given the lambda both the AmazonRDSFullAccess and AmazonRDSDataFullAccess policies without any luck either. You can see by the code below, i've already tried what was recommended in issue #2376.
Not that this should matter, but this lambda is triggered by a Kinesis event trigger.
const AWS = require('aws-sdk');
exports.handler = (event, context, callback) => {
const RDS = new AWS.RDSDataService({apiVersion: '2018-08-01', region: 'us-east-1'})
for (record of event.Records) {
const payload = JSON.parse(new Buffer(record.kinesis.data, 'base64').toString('utf-8'));
const data = compileItem(payload);
const params = {
awsSecretStoreArn: 'arn:aws:secretsmanager:us-east-1:149070771508:secret:xxxxxxxxx,
dbClusterOrInstanceArn: 'arn:aws:rds:us-east-1:149070771508:cluster:xxxxxxxxx',
sqlStatements: `select * from MY_DATABASE.MY_TABLE`
// database: 'MY_DATABASE'
}
console.log('calling executeSql');
RDS.executeSql(params, (error, data) => {
if (error) {
console.log('error', error)
callback(error, null);
} else {
console.log('data', data);
callback(null, { success: true })
}
});
}
}
EDIT: We've run the command through the aws cli and it returns results.
EDIT 2: I'm able to connect to it using the mysql2 package and connecting to it through the URI, so it's defiantly an issue with either the aws-sdk or how I'm using it.
Nodejs excution is not waiting for the result that's why process exit before completing the request.
use mysql library https://www.npmjs.com/package/serverless-mysql
OR
use context.callbackWaitsForEmptyEventLoop =false
Problem was the RDS had to be crated in a VPC, in which the Lambda's were not in