I have an AWS Lambda that runs every five minutes. The Lambda will create a connection to an RDS database inside my VPC and run some queries.
The Lambda will successfully run three times, so for about 15 minutes or so, but, then I get a timeout error:
Task timed out after 600.10 seconds
After this timeout error, the next time the Lambda attempts to run, I can no longer connect to my RDS database. I get the following timeout error:
Error: connect ETIMEDOUT
I'm pretty stumped at this point and could use some more eyes on this:
'use strict';
const mysql = require('mysql');
const util = require('util');
const {
fetchQuery,
insertQuery,
updateQuery,
} = require('./queries');
const {
getInsertValues,
getUpdateValues,
} = require('./utils');
const connection = mysql.createConnection({
host: process.env.DB_HOST,
user: process.env.DB_USER,
password: process.env.DB_PASSWORD,
database: process.env.DB_NAME,
});
// node native promisify
const query = util.promisify(connection.query).bind(connection);
connection.connect();
module.exports.scan = async (event, context, callback) => {
let results = await query(fetchQuery);
console.log(`found ${results.length} forms that are scheduled to be filed`);
if (results.length > 0) {
const insertValues = getInsertValues(results);
const updateValues = getUpdateValues(results);
try {
console.log(`creating user_tasks`);
await query(insertQuery, [insertValues]);
console.log(`updating next_scheduled dates`);
await query(updateQuery, [updateValues]);
callback(null, 'successfully updated form next_scheduled')
} catch (err) {
console.error('error creating user_tasks')
callback(err, 'error creating user_tasks')
}
}
}
Try closing the connection to the database once you are done using it, I previously had a similar error because the open connection was keeping the Lambda alive until it got timeout
Related
I have a firestore database that I want to backup regulalry using google cloud's scheduler alongside google cloud function. I am following this tutorial: https://firebase.google.com/docs/firestore/solutions/schedule-export#gcp-console.
I am using Node.js 16 with 2 GB of RAM with the code below:
const firestore = require('#google-cloud/firestore');
const client = new firestore.v1.FirestoreAdminClient();
const bucket = 'gs://BUCKET_NAME'
exports.scheduledFirestoreExport = (event, context) => {
const databaseName = client.databasePath(
process.env.GCLOUD_PROJECT,
'(default)'
);
return client
.exportDocuments({
name: databaseName,
outputUriPrefix: bucket,
collectionIds: [],
})
.then(responses => {
const response = responses[0];
return response;
})
.catch(err => {
});
};
When I test the function it just returns an Internal Server Error 500. Any ideas?
Also, here is a picture of the errors im getting from the logs:
I'm using GCP PubSub (push subscription to be precisely), and Cloud Run to execute subscribed messages.
Recently I've noticed that Cloud Run executes a same message for five times.
I see there is a retry policy for a subscription which its message has been unhandled (or mishandled), but Cloud Run clearly is giving 200 OK response.
So it seems that sending 200 OK response is not enough, but I cannot find a way to send ack sign properly.
Here's the code to publish message.
function publish(message: string) {
const pubsub = new PubSub({ projectId: 'my-project' });
const dataBuffer = Buffer.from(message);
const topicName = 'my-topic';
pubsub
.topic(topicName)
.publish(dataBuffer)
.then((messageId) => {
console.log(`Message ${messageId} published`);
})
.catch((err) => {
console.log(err);
});
}
publish(JSON.stringify({ foo : 'bar' }));
And here's the code of Cloud Run.
// express app
app.post('/run', async (req, res) => {
try {
const body = req.body.message ? Buffer.from(req.body.message.data, 'base64').toString() : req.body;
// req.body came out to be "{"foo":"bar"}"
const deliveredMessage = JSON.parse(body);
// do something
// Do I have to do something like message.ack() here?
return res.status(200).end();
} catch (e) {
return res.status(502).end();
}
});
These are example responses of Cloud Run retrying.
Here's a graph of unacked messages at that time.
Is your cloud run app running when you did the post?
Because there is a timeout for the messages waiting in the queue, maybe pub/sub is resending the messages before you start to consuming it.
Another thing you can try is putting the response at the first line, like:
app.post('/run', async (req, res) => {
try {
res.status(200)
...
I created 2 tables in RDS aurora:
test-table in aurora db with values id(PK), tasknumber, status , contactid(FK)
contact-table in same db with values contactid(PK), email, phone
I created a trigger in 'test-table' that whenever a 'status' changes, the trigger should call AWS-Lambda ARN.
The Lambda function and Aurora has all the permissions, and security cleared, still on testing from Lambda I get below image error and on updating the 'status' field manually in aurora(via Workbench Sql query) it shows:
Operation failed: There was an error while applying the SQL script to the database.
ERROR 2013: 2013: Lost connection to MySQL server during query
I have attached my Lambda Node.Js code too.
var AWS = require('aws-sdk');
AWS.config.update({ region: 'us-east-1' });
const mysql = require('mysql');
var con = mysql.createConnection({
host: 'correct value',
user: 'root',
password: 'correct value',
port: correct value,
database: 'correct value'
});
exports.handler = (event, context, callback) => {
context.callbackWaitsForEmptyEventLoop = false;
var fk = event.contact_id;
console.log('FOREIGN KEY=', fk)
con.connect(function(err) {
if (err) throw err;
var sql = `SELECT * FROM db.contacts where contact_id=${fk}`
con.query(sql, function(err, result) {
if (err) throw err;
var email = result[0].email;
console.log(email);
var sns = new AWS.SNS();
var params = {
Message: email,
Subject: "Test SNS From Lambda",
TopicArn: "arn:correct value"
};
sns.publish(params, function(err, data) {
if (err) console.log(err, err.stack);
else {
console.log(data);
callback(null, 'Success');
}
});
});
});
};
Image1ErrorMySqlWorkbench
Image2ErrorLambdaaws
Also followed for NodeJs package: https://github.com/isaacs
you have to close the connection if you want the instant response otherwise it will take the default time to close the connection, please let me know if makes sense
I'm trying to connect to AWS documentDB from Lambda function but, not able to connect.
MongoClient.connect never calls the callback function connected.
TLS is off on documentDB Cluster. I'm able to connect via mongo shell.
Lambda & documentDB are in same VPC & Security group.
'use strict';
module.exports.search = async (event, context, callback) => {
const MongoClient = require('mongodb').MongoClient;
const url = "mongodb://xxx:xxxx#xxx-db.cluster-xxx.us-east-2.docdb.amazonaws.com:27017";
console.log("Starting");
MongoClient.connect(url,
{
useNewUrlParser: true
},
function(err, client) {
if(err)
throw err;
console.log("Connected");
db = client.db('mydb');
col = db.collection('mycollection');
col.find({}).toArray().then(result => {
console.log(result);
return { statusCode: 200, body: result };
}).catch(err => {
console.log('=> an error occurred: ', err);
return { statusCode: 500, body: 'error' };
});
});
};
Output only prints starting which was consoled before calling Mongo.Connect.
How to identify or debug the issue ?
Just from looking at the current code I am pretty sure your function exit before it is able to complete. Therefore, your callback is not executed
Because MongoClient.connect runs asynchronously
Try to take a look at some resource around async/await/promise and Lambda
https://medium.com/tensult/async-await-on-aws-lambda-function-for-nodejs-2783febbccd9
How to wait for async actions inside AWS Lambda?
I was not able to connect to MySQL using AWS Lambda with Node.js.
I had tried configured the security groups for AWS MySQL and Lambda. When I used console.log it shows correct response from the data base as the data from db : rk, but when I tried to test it was not showing the correct response.
Below was the logs and the index.js files and logs. Can anybody please guide me ?
index.js (i had updated my code as below ):
var mysql = require('mysql');
var pool = mysql.createPool({
host : 'mydbinstancelamda.connqa9taxeg.us-east-1.rds.amazonaws.com',
user : 'admin',
password : 'password',
database : 'dbname'
});
exports.handler = (event, context, callback)=> {
pool.getConnection(function(err, connection) {
if (err) throw err;
var queryString = "SELECT emp_name from employee where emp_name='rk'";
connection.query(queryString, function(err, rows, fields) {
if (err) throw err;
console.log("the data from db : " + rows[0].emp_name);
callback(null);
connection.release();
});
});
};
error :
Response:
{
"errorMessage": "2018-06-11T02:34:19.817Z ef864d3d-6d1f-11e8-b6e3-97ac89a0f544 Task timed out after 3.00 seconds"
}
Request ID:
"ef864d3d-6d1f-11e8-b6e3-97ac89a0f544"
Function Logs:
START RequestId: ef864d3d-6d1f-11e8-b6e3-97ac89a0f544 Version: $LATEST
dadf1a33-6d22-11e8-869d-7d7e31ccaf6e the data from db : rk
END
Try changing the lambda execution timeout from lambda console as shown in the below picture:
Make sure that security group of RDS MySQL DB is allowing connections from Lambda's security group, if there are the same then you are good to go.
UPDATE:
You need to call callback after the MySQL returns the response.
// change following line:
exports.handler = (event, context, req,res,callback)=> {
// To this line:
exports.handler = (event, context, callback)=> {
After the work is finished, You need to callback to tell lambda that work is complete:
callback(undefined, result);
Example: https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html#nodejs-prog-model-handler-example
i followed my instructions given by #Dilip Kola , were my mistakes are not closing the pool , handler arguments !...
my complete code looks like now :
var mysql = require('mysql');
var pool = mysql.createPool({
host : 'url for mysql ',
user : 'username ',
password : 'paswrod ',
database : 'database-name'
});
exports.handler = (event, context, callback)=> {
pool.getConnection(function(err, connection) {
if (err) throw err;
var queryString = "SELECT emp_name from employee where emp_name='rk'";
connection.query(queryString, function(err, rows, fields) {
if (err) throw err;
console.log("the data from db : " + rows[0].emp_name);
connection.release();
pool.end();
callback(null,rows[0].emp_name);
});
});
};
finally i got my output as :