Using sequelize with AWS Lambda - amazon-web-services

So I'm having some issues trying to use sequelize with AWS Lambda, when querying a table where id = 1 it sometimes returns me data and sometimes it doesn't. I read that Sequelize connections and AWS Lambda service don't get well each other because of the how Lambda executes a function.
My question is, is it not enough to open a connection at the top of a function and then close it at the bottom (before returning something)? If not, what else can I do?
Update:
const findCityByPk = async (pathParameters) => {
const { Postgresql: ps } = require('../libs/utils/potsgresql');
console.log(ps.connection.connectionManager);
const { id } = pathParameters;
try {
const city = await City.findByPk(id);
if (city) {
ps.connection.close();
return {
statusCode: 200,
body: JSON.stringify(city)
};
}
ps.connection.close();
return {
statusCode: 500,
body: JSON.stringify({ message: 'City not found`' })
};
} catch (err) {
console.log(err);
await ps.connection.close();
return {
statusCode: 500,
body: JSON.stringify(err)
};
}
}
This is the code I'm testing it sometimes returns me the correct object from my table
{"id":"1","city_name":"Lima2","zip_code":"12312","time_zone_utc":-5} -> this is what is supposed to be returning, and instead I'm getting this object
{
"requestTime": "06/Dec/2021:18:07:24 +0000",
"requestId": "8b5bf017-c180-41cc-9de6-b07599f0e9b8",
"apiId": "xx",
"resourceId": "xx",
"resourcePath": "/city/{id}",
"path": "/dev/city/1",
"httpMethod": "GET",
"status": "500",
"authLatency": "-",
"integrationLatency": "48",
"integrationStatus": "200",
"responseLatency": "50",
"responseLength": "2",
"errorMessage": "-",
"format": "SLS_ACCESS_LOG",
"version": "1.0.0"
}
And also, this is how it's being made the connection
const createConnection = () => {
console.info("[Postgresql] createConnection: start")
console.info("[Postgresql] createConnection: creating conection start")
let conn;
let string_connection;
try {
string_connection = `postgres://${config.DB_USER}:${config.DB_PASSWORD}#${config.DB_HOST}:5432/${config.DB_NAME}`
//console.debug(`[Postgresql] string_connection: ${string_connection}`)
conn = new Sequelize(string_connection, { logging: false, pool: { max: 1, min: 0, idle: 1000 } });
} catch (e) {
console.debug(`[Postgresql] createConnection: creating conection error ${string_connection}`)
throw e;
}
console.info("[Postgresql] createConnection:creating conection end")
return conn;
}

Related

BatchWriteItemCommand with AWS.DynamoDB class using AWS SDK V3 in Nodejs

I have been trying for hours to perform a DynamoDB DeleteRequest using BatchWriteItemCommand but I keep getting the following error:
Error ValidationException: 1 validation error detected: Value null at 'requestItems.td_notes_sdk.member.1.member.deleteRequest.key' failed to satisfy constraint: Member must not be null
This is what my table looks like:
Partition key: user_id (string)
Sort key: timestamp (number)
DynamoDB Screenshot
This is what my code looks like:
// Import required AWS SDK clients and commands for Node.js
import {
DynamoDBClient,
BatchWriteItemCommand,
} from "#aws-sdk/client-dynamodb";
// Set the parameters
export const params = {
RequestItems: {
"td_notes_sdk": [
{
DeleteRequest: {
Item: {
Key: {
user_id: { S : "bb" },
timestamp: { N : 2 },
},
},
},
},
],
},
};
export const run = async () => {
const ddbClient = new DynamoDBClient({ region: "us-east-2" });
try {
const data = await ddbClient.send(new BatchWriteItemCommand(params));
console.log("Success, items inserted", data);
return data;
} catch (err) {
console.log("Error", err);
}
};
run();
Here are some resources that I've been trying to follow along with:
Resource 1: Writing items in Batch Example
Resource 2: AWS Javascript SDK v3 Documentation
Update: BatchWrite PutRequest work with the code below, so I know that the structure of my keys/attributes is closer to being correct. Still does not work for DeleteRequest.
export const params = {
RequestItems: {
"td_notes_sdk": [
{
PutRequest: {
Item: {
user_id: { "S": "bb" },
timestamp: { "N": "5" },
},
},
},
],
},
};
You don't supply an Item when deleting an item. You supply a Key.
Here is a working example:
const params_delete = {
RequestItems: {
"td_notes_sdk": [
{
DeleteRequest: {
Key: {
user_id: { S: "bb" },
timestamp: { N: "2" },
},
},
},
],
},
};
const delete_batch = async () => {
const ddbClient = new DynamoDBClient({ region: "us-east-2" });
try {
const data = await ddbClient.send(new BatchWriteItemCommand(params_delete));
console.log("Success, item deleted");
return data;
} catch (err) {
console.log("Error", err);
}
};
delete_batch();

Adding data to DynamoDB from the browser (CodePen) fails

I am new to AWS and got the following error when I tried to input data to the dynamodb invoking the lambda function between the 'API gateway' and the 'DynamoDB'.
Error:
Expected params.Item['Age'].S to be a string........
Screenshot of the Error:
Code:
I tried in the browser (CodePen) (I used the correct Invoke URL from the API gateway):
var xhr = new XMLHttpRequest();
xhr.open('POST', 'The API Invoke URL');
xhr.onreadystatechange = function(event){
console.log(event.target.response);
}
xhr.setRequestHeader('Content-Type', 'application/json');
xhr.send(JSON.stringify({age: 26, height: 71, income: 2400}));
The following lambda function is invoked when running the above code from CodePen:
There I have imported the aws-sdk and dynamodb correctly.
exports.fn = (event, context, callback) => {
const params = {
Item: {
"UserId": {
S: "user_" + Math.random()
},
"Age": {
N: event.age
},
"Height": {
N: event.height
},
"Income": {
N: event.income
}
},
TableName: "compare-yourself"
};
dynamodb.putItem(params, function(err, data) {
if (err) {
console.log(err);
callback(err);
} else {
console.log(data);
callback(null, data);
}
});
};
In the above lambda function you can observe that I have formatted the inputs as numbers but in the API gateway, in the POST integration request I have converted the inputs to strings. so the data that is passed via the lambda function is already a string. No need to format by the Lambda function, again.
Body mapper in 'POST Integration-Request':
#set($inputRoot = $input.path('$'))
{
"age" : "$inputRoot.age",
"height": "$inputRoot.height",
"income": "$inputRoot.income"
}
I need to know the reason for the above error and am happy to provide any additional information required.
Thank you in advance.
Change the params to indicate that the value of the age field is "String" and not "Numeric":
const params = {
Item: {
"UserId": {
S: "user_" + Math.random()
},
"Age": {
"S": event.age # This was previously set to "N" which causes the issue
},
"Height": {
N: event.height
},
"Income": {
N: event.income
}
},
TableName: "compare-yourself"
};

AWS XRay Tracing - Not Displaying Trace Map or SubSegments

I am trying to trace a serverless express Lambda function using AWSXRay.
I have tried several different approaches to this and nothing seems to work.
If I leave out the aws-xray-sdk-express middleware, I will see all my segments in the Timeline, and see my Lambda function appearing twice in the Trace Map. If I include the express middleware, I will see the middleware segment ('Super Dooper Trace Function') in the Trace Map and just the first subsegment ('MyFirstTrace') in the Timeline (not in the Trace Map).
I am trying to get all Subsegments to appear in both the Timelines and Node Graphs
const AWSXray = require('aws-xray-sdk');
const xrayExpress = require('aws-xray-sdk-express');
const express = require('express');
const awsServerlessExpress = require('aws-serverless-express');
module.exports.handler = async (event, context) => {
const app = express();
app.use(xrayExpress.openSegment('Super Dooper Trace Function'));
app.get('/test', async (req, res) => {
const result = await doWork();
res.send(result);
});
app.use(xrayExpress.closeSegment());
const server = awsServerlessExpress.createServer(app);
return awsServerlessExpress.proxy(server, event, context, 'PROMISE').promise;
}
const doWork = async () => {
const res1 = await traceMyFunction('MyFirstTrace', 3000)
const res2 = await traceMyFunction('MySecondTrace', 3000);
const res3 = await traceMyFunction('MyThirdTrace', 2000);
return [res1, res2, res3];
}
const traceMyFunction = (name, delayMs) => {
return trace(name, async () => {
return delay(delayMs)
});
}
function trace(name, promiseFunction) {
return new Promise((resolve, reject) => {
AWSXray.captureAsyncFunc(name, (subSegment) => {
promiseFunction(subSegment)
.then((result) => {
resolve(result);
subSegment.close();
}).catch((e) => {
subSegment.close();
reject(e)
});
});
});
}
const delay = (ms) => {
return new Promise((res, rej) => {
setTimeout(() => res({ time: ms }), ms)
});
};
This is the resultant XRay Trace
And this is the Trace Raw Data
{
"Duration": 8.278,
"Id": "1-5e2f6cc3-a99e6c08f02ce0aec7ab7121",
"Segments": [
{
"Document": {
"id": "87ca46b60ded68e1",
"name": "Super Dooper Trace Function",
"start_time": 1580166338.92,
"end_time": 1580166347.198,
"http": {
"request": {
"url": "http://localhost/test",
"method": "GET",
"user_agent": "",
"client_ip": ""
},
"response": {
"status": 200
}
},
"aws": {
"xray": {
"sdk": "X-Ray for Node.js",
"sdk_version": "2.5.0",
"package": "aws-xray-sdk"
}
},
"service": {
"version": "unknown",
"runtime": "node",
"runtime_version": "v12.13.0",
"name": "unknown"
},
"trace_id": "1-5e2f6cc3-a99e6c08f02ce0aec7ab7121",
"subsegments": [
{
"id": "98e9f32273700e6e",
"name": "MyFirstTrace",
"start_time": 1580166339.078,
"end_time": 1580166342.082
}
]
},
"Id": "87ca46b60ded68e1"
}
]
}
Serverless Express has known incompatabilities with the current X-Ray SDK - due to the fact Lambda generates its own Segment, then the Express middleware tries to create one as well. We're planning to address this in the near future.
https://github.com/aws/aws-xray-sdk-node/issues/45
For the full explanation, see this thread: https://github.com/aws/aws-xray-sdk-node/issues/30

Searching DynamoDB for non primary keys and integrating into Alexa Skills

I am trying to search a non primary key using AWS Lambda and integrating it into the Alexa Skills Kit. I am very new to using DynamoDB and Alexa Skills Kit and I'm struggling to find any solutions to this online. The basic premise for what I am trying to do is querying the table yesno with two columns, id and message. Only looking through the message column to find a match with the text i specify in params.
Here is the Lambda code I am working with:
const AWSregion = 'eu-west-1';
const Alexa = require('alexa-sdk');
const AWS = require('aws-sdk');
//params for searching table
const params = {
TableName: 'yesno',
Key:{ "message": 'Ben Davies' }
};
AWS.config.update({
region: AWSregion
});
exports.handler = function(event, context, callback) {
var alexa = Alexa.handler(event, context);
// alexa.appId = 'amzn1.echo-sdk-ams.app.1234';
// alexa.dynamoDBTableName = 'YourTableName'; // creates new table for session.attributes
alexa.registerHandlers(handlers);
alexa.execute();
};
const handlers = {
'LaunchRequest': function () {
this.response.speak('welcome to magic answers. ask me a yes or no question.').listen('try again');
this.emit(':responseReady');
},
'MyIntent': function () {
var MyQuestion = this.event.request.intent.slots.MyQuestion.value;
console.log('MyQuestion : ' + MyQuestion);
readDynamoItem(params, myResult=>{
var say = MyQuestion;
say = myResult;
say = 'you asked, ' + MyQuestion + '. I found a reckord for: ' + myResult;
this.response.speak(say).listen('try again');
this.emit(':responseReady');
});
},
'AMAZON.HelpIntent': function () {
this.response.speak('ask me a yes or no question.').listen('try again');
this.emit(':responseReady');
},
'AMAZON.CancelIntent': function () {
this.response.speak('Goodbye!');
this.emit(':responseReady');
},
'AMAZON.StopIntent': function () {
this.response.speak('Goodbye!');
this.emit(':responseReady');
}
};
// END of Intent Handlers {} ========================================================================================
// Helper Function =================================================================================================
function readDynamoItem(params, callback) {
var AWS = require('aws-sdk');
AWS.config.update({region: AWSregion});
var dynamodb = new AWS.DynamoDB();
console.log('reading item from DynamoDB table');
dynamodb.query(params, function (err, data) {
if (err) console.log(err, err.stack); // an error occurred
else{
console.log(data); // successful response
callback(data.Item.message);
}
});
}
I know I am probably doing this completely wrong but there isn't much online for integrating DynamoDB with an Alexa Skill and the only thing i was able to find was searching by ID. This doesn't work for what i want to do without pulling all the items from the table into a map or a list, and seeing as I want to create a big database it seems quite inefficient.
On the Alexa side of things I am receiving the following service request when testing the code:
{
"session": {
"new": true,
"sessionId": "SessionId.f9558462-6db8-4bf5-84aa-22ee0920ae95",
"application": {
"applicationId": "amzn1.ask.skill.9f280bf7-d506-4d58-95e8-b9e93a66a420"
},
"attributes": {},
"user": {
"userId": "amzn1.ask.account.AF5IJBMLKNE32GEFQ5VFGVK2P4YQOLVUSA5YPY7RNEMDPKSVCBRCPWC3OBHXEXAHROBTT7FGIYA7HJW2PMEGXWHF6SQHRX3VA372OHPZZJ33K7S4K7D6V3PXYB6I72YFIQBHMJ4QGJW3NS3E2ZFY5YFSBOEFW6V2E75YAZMRQCU7MNYPJUMJSUISSUA2WF2RA3CIIDCSEY35TWI"
}
},
"request": {
"type": "IntentRequest",
"requestId": "EdwRequestId.7310073b-981a-41f8-9fa5-03d1b28c5aba",
"intent": {
"name": "MyIntent",
"slots": {
"MyQuestion": {
"name": "MyQuestion",
"value": "erere"
}
}
},
"locale": "en-US",
"timestamp": "2018-01-25T14:18:40Z"
},
"context": {
"AudioPlayer": {
"playerActivity": "IDLE"
},
"System": {
"application": {
"applicationId": "amzn1.ask.skill.9f280bf7-d506-4d58-95e8-b9e93a66a420"
},
"user": {
"userId": "amzn1.ask.account.AF5IJBMLKNE32GEFQ5VFGVK2P4YQOLVUSA5YPY7RNEMDPKSVCBRCPWC3OBHXEXAHROBTT7FGIYA7HJW2PMEGXWHF6SQHRX3VA372OHPZZJ33K7S4K7D6V3PXYB6I72YFIQBHMJ4QGJW3NS3E2ZFY5YFSBOEFW6V2E75YAZMRQCU7MNYPJUMJSUISSUA2WF2RA3CIIDCSEY35TWI"
},
"device": {
"supportedInterfaces": {}
}
}
},
"version": "1.0"
}
And I am receiving a service response error simply saying 'The response is invalid'
Any help with this would be greatly appreciated
I would like to help you in dynamo db part.
In order to access non primary key columns in dynamodb you should perform scan operation.
For your table (yesno), id is a primary key and message is an additional column.
Snippet to access non primary key column [Message]
var dynamodb = new AWS.DynamoDB();
var params = {
TableName: 'yesno',
FilterExpression: 'message = :value',
ExpressionAttributeValues: {
':value': {"S": "Ben Davies"}
}
};
dynamodb.scan(params, function(err, data) {
if (err) // an error occurred
else console.log(data); // successful response
});
Snippet to access primary key column [Id]
var docClient = new AWS.DynamoDB.DocumentClient();
//Get item by key
var params = {
TableName: 'sis_org_template',
Key: { "id": "1"}
};
docClient.get(params, function(err, data) {
if (err) // an error occurred
else console.log(data); // successful response
});

before Remote in Loopback

I'm trying to use the before Remote hook in loopback to check an user token before the call of my remote method, but the thing is that I can only return a javascript Error, like these.
"error": {
"statusCode": 401,
"name": "Error",
"message": ""
}
This is the code that i'm using in the before remote.
Model.beforeRemote('method', function (context, unused, next) {
let token = Model.app.models.Token;
let id = context.args.Id;
let date = moment();
Rx.Observable.fromPromise(token.find({
where: {
and: [
{ id: id, },
{ expiration: { gt: date } }
]
}
})).subscribe((token => {
if (token.length > 0) {
next();
} else {
let err = new Error();
err.status = 401;
delete err.stack;
return next()
}
}))
});
And I need a "custom" response that isn't an error, something like these.
{
"success": false,
"data": {
"service": "self",
"operation": "rest",
"code": "unauthorized"
},
"message": "Invalid token"
}
I tried with the after Remote hook and I can change the response to get something like that, but I want to get a quicker response in the case that the token is invalid.
Is there any way to achieve this with the before hook? or I had to use after hook?
Thanks
You may want to pass the error to the next function:
return next(err)