AWS Lambda function timing out - amazon-web-services

In my local mocha tests the following handler function works just fine. However, when I upload to AWS (using Serverless framework) it times out (unless you don't provide a uid parameter where it then correctly responds immediately).
What's particularly odd is that in less than 3 seconds (timeout is set at 5 seconds), the job completes and even the "post-facto" log message is output but it somehow calling the callback and that is not completing the Lambda function
Here's the cloudwatch log:
]1
And here's the handler function:
export const handler = (event: IRequestInput, context: IContext, cb: IGatewayCallback) => {
console.log('EVENT:\n', JSON.stringify(event, null, 2));
const uid = _.get(event, 'queryStringParameters.uid', undefined);
if(!uid) {
cb(null, {
statusCode: 412,
body: 'no User ID was provided by frontend'
});
return;
}
oauth.getRequestToken()
.then(token => {
console.log('Token is:\n', JSON.stringify(token, null, 2));
console.log('User ID: ', uid);
token.uid = uid;
return Promise.resolve(token);
})
.then((token) => {
console.log('URL: ', token.url);
cb(null, {
statusCode: 200,
body: token.url
});
console.log('post-facto');
})
.catch((err: PromiseError) => {
console.log('Problem in getting promise token: ', err);
cb(err.message);
});
};

Add the following as the first line of your handler function:
context.callbackWaitsForEmptyEventLoop = false

I guess that you're using lambda with "Node.js Runtime 0.10"
So you should add
context.done(null, 'Terminate Lambda');
to terminate the execution.
As the AWS lambda document, it mentions that:
The callback is supported only in the Node.js runtime v4.3. If you
are using the earlier runtime v0.10.42, you need to use the context
methods (done, succeed, and fail) to properly terminate the Lambda
function.
Please refer this link for above information

Related

Lambda connect to Dynamodb

I have lambda function like below
// Loads in the AWS SDK
const AWS = require('aws-sdk');
// Creates the document client specifing the region
const ddb = new AWS.DynamoDB.DocumentClient({region: 'us-east-1'});
exports.handler = async (event, context, callback) => {
// Handle promise fulfilled/rejected states
await readMessage().then(data => {
data.Items.forEach(function(item) {
console.log(item.message)
});
callback(null, {
// If success return 200, and items
statusCode: 200,
body: data.Items,
headers: {
'Access-Control-Allow-Origin': '*',
},
})
}).catch((err) => {
// If an error occurs write to the console
console.error(err);
})
};
// Function readMessage
// Reads 10 messages from the DynamoDb table Message
// Returns promise
function readMessage() {
const params = {
TableName: 'Message',
Limit: 10
}
return ddb.scan(params).promise();
}
my dynamo db is in region us-east-1 , but above code is not running , it is just timing out.But i am able to run simple hello world inside handler function , and it just works fine.
I have enabled VPC with 6 subnets.. also disabled VPC , not matter what it is timing out without any error logs.. I even increased timeout to 5 mins , it did not do anything.Should I install aws-sdk or something ? Sorry I am completely new to lambda.

AWS Lambda function invoked from another Lambda function runs stale request in queue

I have a Lambda function (A) that calls another Lambda function (B) and returns while the other one is still executing. Everything works perfectly locally (using sls offline), but when I deploy it, the following behavior occurs:
I call A for the first time. I see on Cloudwatch that A runs successfully and returns, but nothing on B, which apparently doesn't run.
I call A another time. Cloudwatch now shows that both A and B run, but the logs show that the invocation of B had request parameters corresponding to the first run of A.
Ensuing runs of A result in runs of B with request params from the previous run of A (e.g. B always lags behind by one and never runs the current invocation like it is supposed to, almost like each call of A pushes out the previous B invocation from a queue of size 1)
Some additional info:
No VPC's are involved (if that makes a difference?)
I am using sls
Anyone have any idea why this is happening? I know that AWS async invocations go into a queue, but I'm pretty sure they should just run ASAP if there's nothing in the queue.
Here's the code:
module.exports.A = async (event, context) => {
context.callbackWaitsForEmptyEventLoop = false;
try {
... some code ...
await connectToDatabase();
... some code ...
const newReq = req;
newReq['newSessionId'] = savedSession._id.toString();
const params = {
FunctionName: config.apiName + '-' + config.apiVersion + '-B',
InvocationType: 'Event',
Payload: JSON.stringify(newReq)
};
lambda.invoke(params, (err, res) => {
if (err) {
console.log(err);
} else {
console.log(res);
}
});
return {
statusCode: 200,
headers: {
'Access-Control-Allow-Origin' : '*',
'Access-Control-Allow-Credentials' : true
},
body: JSON.stringify(savedSession)
}
} catch (err) {
console.log(err);
return {
statusCode: err.statusCode || 500,
headers: { 'Content-Type': 'text/plain' },
body: 'Could not create session.'
}
}
}
module.exports.B = async (event, context) => {
context.callbackWaitsForEmptyEventLoop = false;
try {
... some code ....
// Connect to Atlas
await connectToDatabase();
... some code ...
for (let i = 0; i < applicants.length; i++) {
... some code ...
const savedResume = await DownloadResume(resumeId, accessToken, fileName, oauthSrcs.GSCRIPT);
conversionPromises.push(ConvertAndExportToS3(fileName, savedResume, sessionKey));
}
await Promise.all(conversionPromises);
... some code ...
return {
statusCode: 200,
headers: {
'Access-Control-Allow-Origin' : '*',
'Access-Control-Allow-Credentials' : true
},
body: JSON.stringify('Converted all resumes successfully!')
}
} catch (err) {
console.log(err);
return {
statusCode: err.statusCode || 500,
headers: { 'Content-Type': 'text/plain' },
body: 'Could not create session.'
}
}
}
I see you are returning from A without waiting for lambda.invoke to finish. I think you should.
To clarify, you are indeed invoking an asynchronous lamdda (meaning you don't have to wait for the child lambda to finish), but the invocation itself is synchronous. Not sure what would happen if you returns A in the middle of B's invocation.
Also, did you wait long enough so you are sure that the 2nd run of A triggers anything ? It could be that a cold start on B makes it look that way if you don't wait long enough on first call

Rest-API call fails in lambda function

I'm new about lambda functions in AWS and I need some suggestions to figure out the nature of the problem.
AWS Lambda function based on Javascript using node.js 12.x.
I did set up local development environment based on SAM (sam cli/aws cli/docker/IntelliJ) on Ubuntu 18.04 and MacOs Catalina and a simple basic lambda function work, on both systems.
I can set up logs and see them on via IntelliJ when docker runs.
The function was created using sam init command from a terminal and selecting a simple hello world.
I did add a Rest-API call in it.
Nothing fancy, using request 2.88.2 (I know is deprecated and I did try to use other ways, all of them fails anyway so I'm stick with request for now).
Basically what is happening is that the call to the API "seems" not happening at all.
Logs placed before and after the API call are showing up.
Logs inside the API call, like to show the errors or results, are never showing up.
So far only in one case I was able to see an error message coming from the API, when I removed the URI.
And as expected the API returned an error message saying : invalid URI.
Otherwise NOTHING. Here some code.
This function is called from the lambda handler.
function getToken() {
const request = require('request');
const qs = require('querystring');
console.log("getToken function called");
let bodyData = qs.stringify({
username: 'test',
password: 'xxxxx',
grant_type: 'password'
});
console.log("getToken bodyData : " + bodyData);
let options = {
url: "https://blahblahblah/function",
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
'Authorization': 'Basic xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
},
form: bodyData
};
console.log("getToken options : " + options);
var request1 = request(options, function(err, response, body) {
console.log("returned from API call");
if (err) {
console.log("Error from getToken : " + err);
return null;
} else {
console.log("Answer from getToken : " + body);
return body;
}
});
}
I did test the connection and API using Postman and is working.
All the logs inside the request are NEVER coming up.
No matter changing options (did try many many different ways).
What am I doing wrong ?
Any suggestion on how to track this problem ?
Thanks
STeve
This is the correct behaviour. Because the function getToken is not waiting for the http request to complete. you should either convert the function to use promise/async-await or simply callback.
Promise
async function getToken() {
...
return new Promise((resolve, reject) => {
request(options, function (err, response, body) {
console.log("returned from API call");
if (err) {
console.log("Error from getToken : " + err);
resolve(null);
} else {
console.log("Answer from getToken : " + body);
resolve(body);
}
});
});
}
// Then in the lambda Handler:
// await getToken()
Callback
function getToken(callback) {
...
request(options, function (err, response, body) {
console.log("returned from API call");
if (err) {
console.log("Error from getToken : " + err);
callback(null);
} else {
console.log("Answer from getToken : " + body);
callback(body);
}
});
}
// Then in the lambda
getToken(() => {
// handle the http request response
})

Serverless Lambda invoking Lambda. Getting no errors but a empty response

Now day we can use the AWS StepFunctions if we want to make an lambda function to call another one.
But for now I need do support the code in production that was written before the StepFunctions time.
For that reason I need to understand how it works. I was trying to create a very simple lambda calling another lambda function trough AWS-SDk.
I have the follow serverless.yml
service: lambdaCallLambda
provider:
name: aws
runtime: nodejs6.10
functions:
hello:
handler: handler.hello
funcOne:
handler: handler.funcOne
funcTwo:
handler: handler.funcTwo
#Must install aws-sdk. #npm install --save aws-sdk
And this is the handler.js:
'use strict';
//https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Lambda.html
var Lambda = require('aws-sdk/clients/lambda');
module.exports.hello = (event, context, callback) => {
const response = {
statusCode: 200,
body: JSON.stringify({
message: 'hello',
input: event,
}),
};
callback(null, response);
};
module.exports.funcOne = (event, context, callback) => {
var text='';
var i = 0;
for (i = 0; i < 5; i++) {
text += "The number is " + i + "\n";
}
console.log(text);
//https://docs.aws.amazon.com/general/latest/gr/rande.html
const lambda = new Lambda({
region: 'us-east-1'
});
console.log('control 3');
/*
https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Lambda.html#constructor-property
To invoke a Lambda function
This operation invokes a Lambda function
https://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html
Payload - JSON that you want to provide to your Lambda function as input.
*/
var params = {
ClientContext: "lambdaCallLambda",
FunctionName: "lambdaCallLambda-dev-funcOne",
InvocationType: "Event",
LogType: "Tail",
Payload: '{"jsonKey2":123}',
Qualifier: "1"
};
lambda.invoke(params, function(err, data) {
if (err){
console.log('control error\n');
console.log(err, err.stack); // an error occurred
}
else{
console.log('control OK\n');
console.log(data); // successful response
}
/*
data = {
FunctionError: "",
LogResult: "",
Payload: <Binary String>,
StatusCode: 123
}
*/
});
};
module.exports.funcTwo = async (event, context) => {
return 2;
//return '{"funcTwo":20000}';
//console.log("funcTwo = " + event);
};
After deploy sls deploy and call funcOne I get this 2 outputs:
LOCAL:
sls invoke local --function funcOne
Serverless: INVOKING INVOKE
The number is 0
The number is 1
The number is 2
The number is 3
The number is 4
control 3
control OK
{ StatusCode: 202, Payload: '' }
Invoking remotely in AWS:
sls invoke --function funcOne
{
"errorMessage": "Unexpected token (",
"errorType": "SyntaxError",
"stackTrace": [
" ^",
"SyntaxError: Unexpected token (",
"createScript (vm.js:56:10)",
"Object.runInThisContext (vm.js:97:10)",
"Module._compile (module.js:542:28)",
"Object.Module._extensions..js (module.js:579:10)",
"Module.load (module.js:487:32)",
"tryModuleLoad (module.js:446:12)",
"Function.Module._load (module.js:438:3)",
"Module.require (module.js:497:17)",
"require (internal/module.js:20:19)"
]
}
Error --------------------------------------------------
Invoked function failed
For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.
Get Support --------------------------------------------
Docs: docs.serverless.com
Bugs: github.com/serverless/serverless/issues
Issues: forum.serverless.com
Your Environment Information -----------------------------
OS: linux
Node Version: 8.11.3
Serverless Version: 1.29.2
Does someone knows hat is happening here? Specially for the first scenario where I dont have any error.
This is what I get from the documentation
Parameters:
err (Error) — the error object returned from the request. Set to null if the request is successful.
data (Object) — the de-serialized data returned from the request. Set to null if a request error occurs. The data object has the following properties:
Status — (Integer)
It will be 202 upon success.
Update
After Eduardo Díaz suggestion -
I have changed lambda.invoke to:
lambda.invoke({
FunctionName: 'lambdaCallLambda-dev-funcOne',
Payload: JSON.stringify(event, null, 2)
}, function(error, data) {
if (error) {
console.log('control ErrorFoncOne\n');
context.done('error', error);
}
if(data.Payload){
console.log('control SuccessFoncOne\n');
context.succeed(data)
}
});
And this what I get for Local and Remote:
{
"errorMessage": "Unexpected token (",
"errorType": "SyntaxError",
"stackTrace": [
"Module.load (module.js:487:32)",
"tryModuleLoad (module.js:446:12)",
"Function.Module._load (module.js:438:3)",
"Module.require (module.js:497:17)",
"require (internal/module.js:20:19)"
]
}
It is a SyntaxError. There is a "(" somewhere.
I have found another developer with the same error here.
Note:
No error logs in CloudWatch
Try to send the event in the payload:
lambda.invoke({
FunctionName: 'name_lambda_function',
Payload: JSON.stringify(event, null, 2)
}, function(error, data) {
if (error) {
context.done('error', error);
}
if(data.Payload){
context.succeed(data.Payload)
}
});
I strongly suspect this problem is rooted in your handler signature for funcTwo:
module.exports.funcTwo = async (event, context) => {
NodeJS 6.10 does not support async/await. Node always complains about the token after the async token, for whatever reason. If you don't use a fat arrow function:
module.exports.funcTwo = async function(event, context) {
Node will complain: Unexpected token function.
Options
Deploy the function to NodeJS 8.10 instead.
Get rid of the async keyword in the handler signature.
Use a build tool (like serverless-webpack) to transpile the function down to ES6 (or lower).
Note: If you stick with the 6.10 runtime, I think you'll want to do something like context.succeed(2); or callback(null, 2);, rather than return 2;. Simply using a return statement does seem to work on 8.10.
I have found the issue. We need JSON.stringfy in the playload in lamdda.invoke method. No extra parameters are needed. Only the JSON as per the documentation.
lambda.invoke({
FunctionName: 'lambdaCallLambda-dev-funcTwo',
Payload: JSON.stringify({"jsonKey2":i})
...
From the AWS documentation for playload we have:
Payload
JSON that you want to provide to your Lambda function as input.
Note:
the async in front of the functions
lambda.invoke({
FunctionName: 'lambdaCallLambda-dev-funcTwo',
Payload: JSON.stringify({"jsonKey2":i})
}, async function(error, data) { ...
and
module.exports.funcTwo = async(event, context, callback) => { ...
Gives me this output:
Loop nb=1
Loop nb=2
Loop nb=3
Loop nb=4
Loop nb=5
{"message":"hello from funcTwo","event":{"jsonKey2":3}}
{"message":"hello from funcTwo","event":{"jsonKey2":2}}
{"message":"hello from funcTwo","event":{"jsonKey2":1}}
{"message":"hello from funcTwo","event":{"jsonKey2":5}}
{"message":"hello from funcTwo","event":{"jsonKey2":4}}
While the absence of async gives me:
Loop nb=1
Loop nb=2
Loop nb=3
Loop nb=4
Loop nb=5
{"message":"hello from funcTwo","event":{"jsonKey2":1}}
{"message":"hello from funcTwo","event":{"jsonKey2":2}}
{"message":"hello from funcTwo","event":{"jsonKey2":3}}
{"message":"hello from funcTwo","event":{"jsonKey2":4}}
{"message":"hello from funcTwo","event":{"jsonKey2":5}}
I'm going to share the handle.js code just in case someone else needs it:
'use strict';
//https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Lambda.html
var Lambda = require('aws-sdk/clients/lambda');
module.exports.hello = (event, context, callback) => {
const response = {
statusCode: 200,
body: JSON.stringify({
message: 'hello',
input: event,
}),
};
callback(null, response);
};
/*
https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Lambda.html#constructor-property
To invoke a Lambda function
This operation invokes a Lambda function
https://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html
Payload - JSON that you want to provide to your Lambda function as input.
Serverless Framework: Lambdas Invoking Lambdas
https://lorenstewart.me/2017/10/02/serverless-framework-lambdas-invoking-lambdas/
How to escape async/await hell
https://medium.freecodecamp.org/avoiding-the-async-await-hell-c77a0fb71c4c
Iterating a Loop Using Lambda
https://docs.aws.amazon.com/step-functions/latest/dg/tutorial-create-iterate-pattern-section.html
AWS Lambda “Process exited before completing request”
https://stackoverflow.com/questions/31627950/aws-lambda-process-exited-before-completing-request
Class: AWS.Lambda
https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Lambda.html#constructor-property
Invoke
https://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html#API_Invoke_RequestSyntax
Programming Model(Node.js)
https://docs.aws.amazon.com/lambda/latest/dg/programming-model.html
AWS Lambda Examples
https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/lambda-examples.html
*/
//The event is a file inserted in S3. This function funcOne reads the file and loop trough all quantities of products.
//Invoking a lamda function funcTwo for each process in the loop.
module.exports.funcOne = (event, context, callback) => {
//https://docs.aws.amazon.com/general/latest/gr/rande.html
const lambda = new Lambda({
region: 'us-east-1'
});
//Loop
//nbProducts = loop trough the products JSON list in S3
var nbProducts=5;
for (let i = 1; i <= nbProducts; i++) {
console.log('Loop nb='+i+'\n');
lambda.invoke({
FunctionName: 'lambdaCallLambda-dev-funcTwo',
Payload: JSON.stringify({"jsonKey2":i})
}, async function(error, data) {
if (error) {
//console.log('control ErrorFoncOne\n');
context.done('error', error);
}
if(data.Payload){
//console.log('control SuccessFoncOne\n');
console.log(data.Payload);
//context.succeed(data)
}
});
}
};
module.exports.funcTwo = async(event, context, callback) => {
callback(null, { message: 'hello from funcTwo', event });
};
just a change of one line would get you the respose
do change in params
'use strict';
//https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Lambda.html
var Lambda = require('aws-sdk/clients/lambda');
module.exports.hello = (event, context, callback) => {
const response = {
statusCode: 200,
body: JSON.stringify({
message: 'hello',
input: event,
}),
};
callback(null, response);
};
module.exports.funcOne = (event, context, callback) => {
var text='';
var i = 0;
for (i = 0; i < 5; i++) {
text += "The number is " + i + "\n";
}
console.log(text);
//https://docs.aws.amazon.com/general/latest/gr/rande.html
const lambda = new Lambda({
region: 'us-east-1'
});
console.log('control 3');
var params = {
ClientContext: "lambdaCallLambda",
FunctionName: "lambdaCallLambda-dev-funcOne",
InvocationType: "RequestResponse", /* changed this line from
"Event" to "RequestResponse"*/
LogType: "Tail",
Payload: '{"jsonKey2":123}',
Qualifier: "1"
};
lambda.invoke(params, function(err, data) {
if (err){
console.log('control error\n');
console.log(err, err.stack); // an error occurred
}
else{
console.log('control OK\n');
console.log(data); // successful response
}
/*
data = {
FunctionError: "",
LogResult: "",
Payload: <Binary String>,
StatusCode: 123
}
*/
});
};
module.exports.funcTwo = async (event, context) => {
return 2;
//return '{"funcTwo":20000}';
//console.log("funcTwo = " + event);
};

AWS - Sending 1000's of emails from Lambda / Node.js

I have a "main" Lambda function that gets triggered by SNS. It pulls a list of recipients from the database and it needs to send each of them a message based on a template, replacing things like first name and such.
The way I have it setup is I created another Lambda function called "email-send" which is subscribed to "email-send" topic. The "main" Lambda then loops through the recipients list and publishes messages to "email-send" with a proper payload (from, to, subject, message). This might eventually need to process 1000's of emails in a single batch.
Is this a good approach to my requirements? Perhaps Lambda/SNS is not a way to go? If so, what would you recommend.
With this setup I am running into issues when my "main" function finishes running and somehow "sns.publish" does not get triggered in my loop. I assume because I am not letting it finish. But I am not sure how to fix it, being a loop.
Here is the snippet from my Lambda function:
exports.handler = (event, context, callback) => {
// code is here to pull data into "data" array
// process records
for (var i = 0; i < data.length; i++) {
var sns = new aws.SNS();
sns.publish({
Message: JSON.stringify({ from: data[i].from, to: data[i].to, subject: subject, body: body }),
TopicArn: 'arn:aws:sns:us-west-2:XXXXXXXX:email-send'
}, function(err, data) {
if (err) {
console.log(err.stack);
} else {
console.log('SNS pushed!');
}
});
}
context.succeed("success");
};
Thanks for any assistance.
Your code is doing this...
Begin calling sns.publish() 1000 times
Return (through context.succeed())
You didn't wait for those 1000 calls to finish!
What your code should do is...
Begin calling sns.publish() 1000 times
When all calls to sns.publish() has returned, then return. (context.succeed is old so we should use callback() instead).
Something like this...
// Instantiate the client only once instead of data.length times
const sns = new aws.SNS();
exports.handler = (event, context, callback) => {
const snsCalls = []
for (var i = 0; i < data.length; i++) {
snsCalls.push(sns.publish({
Message: JSON.stringify({
from: data[i].from,
to: data[i].to,
subject: subject,
body: body
}),
TopicArn: 'arn:aws:sns:us-west-2:XXXXXXXX:email-send'
}).promise();
}
return Promise.all(snsCalls)
.then(() => callback(null, 'Success'))
.catch(err => callback(err));
};
I think that a better approach is using AWS Lambda API.
That way, you don't need SNS.
For example:
var lambda = new AWS.Lambda({region: AWS_REGION});
function invokeWorkerLambda(task, callback) {
var params = {
FunctionName: WORKER_LAMBDA_NAME,
InvocationType: 'Event',
Payload: JSON.stringify({.....})
};
lambda.invoke(params, function(err, data) {
if (err) {
console.error(err, err.stack);
callback(err);
} else {
callback(null, data);
}
});
}
As you can see, you don't need SNS for lambda function's invocation.
Important: Another suggestion is to create an Array of invocations (functions) and later execute them as follow:
async.parallel(invocations, function(err) {
if (err) {
console.error(err, err.stack);
callback(err);
}
});
Take a look at this link where I got a lot of knowledge about Lambda invocation: https://cloudonaut.io/integrate-sqs-and-lambda-serverless-architecture-for-asynchronous-workloads/