----------------- GIVING MORE CONCRETE EXAMPLES BELOW -----------------
I have the following function, where I take user, name, sex from parameters and update the existing client for that user.
function createRecord(user, personName, sex) {
var sexPayload = [];
var dynamoParams = {
TableName: 'account',
Key: {
id: user
},
UpdateExpression: "set #client.#person.#sex = :sexPayload",
ExpressionAttributeNames: {
"#client": "client",
"#person": personName,
"#sex": sex
},
ExpressionAttributeValues: {
":sexPayload": sexPayload,
},
ConditionalExpression: "attribute_not_exists(#client.#person.#sex)"
};
docClient.update(dynamoParams, function(err, data){
something...
});
}
if personName = "John" sex = "man",
this creates
client: {
John: {"M": {"man": {"L": []}}},
}
and when I pass in another name = "John" sex = "female",
this overrides existing John "man" and writes "female" under John
client: {
John: {"M": {"female": {"L": []}}},
}
What I want to achieve is this:
client: {
John: {"M": {"man": {"L": []}, {"female": {"L": []}},
}
what am I doing wrong??
Related
I am able to SET values for reserved keywords but not able to remove certain keywords in dynamodb.
My expression is below. Not able to remove data if I give like this. But able to remove size though.
Invalid UpdateExpression: Attribute name is a reserved keyword; reserved keyword:data
{
ConditionExpression: "#id = :id",
ExpressionAttributeNames: {
"#id": "id",
“#name": “name",
},
ExpressionAttributeValues: {
":id": “1234566",
“:name": “John",
},
Key: {
id: "1234566",
},
ReturnValues: "ALL_NEW",
TableName: “table_name",
UpdateExpression: "SET #name = :name REMOVE size, data",
}
Modified to include data in expression attribute names as below. Still throws error.
{
ConditionExpression: "#id = :id",
ExpressionAttributeNames: {
"#id": "id",
“#name": “name",
“#size": “size",
“#data": “data",
},
ExpressionAttributeValues: {
":id": “1234566",
“:name": “John",
“:size": undefined,
“:data": undefined,
},
Key: {
id: "1234566",
},
ReturnValues: "ALL_NEW",
TableName: "table_name",
UpdateExpression: "SET #name = :name, #size = :size, #data =
:data REMOVE size, data”,
}
Remove size and data from ExpressionAttributeValues and from SET UpdateExpression. Add #size and #data to REMOVE on UpdateExpression instead of size and data.
{
ConditionExpression: "#id = :id",
ExpressionAttributeNames: {
"#id": "id",
“#name": “name",
“#size": “size",
“#data": “data",
},
ExpressionAttributeValues: {
":id": “1234566",
“:name": “John"
},
Key: {
id: "1234566",
},
ReturnValues: "ALL_NEW",
TableName: "table_name",
UpdateExpression: "SET #name = :name, REMOVE #size, #data”,
}
Reserved words in DynamoDB
to do that dynamically you need only to provide a json object with this function
it's create dynamically
-ExpressionAttributeNames
-ExpressionAttributeValues
-UpdateExpression
also skip reserved keyword for dynamodb
def generate_update_att(body):
"""
#############input
body={
'newval':newData['newval'],
'lastname':newData['lastname'],
'name':newData['name']
}
#################output
-ExpressionAttributeNames
-ExpressionAttributeValues
-UpdateExpression
"""
UpdateExpression = ["set "]
ExpressionAttributeValues = dict()
ExpressionAttributeNames=dict()
for key, val in body.items():
UpdateExpression.append(f" #{key} = :{key},")
ExpressionAttributeValues[f":{key}"] = val
ExpressionAttributeNames[f"#{key}"] = key
return "".join(UpdateExpression)[:-1], ExpressionAttributeValues,ExpressionAttributeNames
function Call
my_data={
'newval':newData['newval'],
'lastname':newData['lastname'],
'name':newData['name']
}
a, v,z = generate_update_att(my_data)
my_table.update_item(
Key={'id':my_id},
UpdateExpression=my_UpdateExpression,
ExpressionAttributeValues=dict(my_ExpressionAttributeValues),
ExpressionAttributeNames=dict(my_ExpressionAttributeNames)
)
I have a lambda function which should update the field on dynamodb using AppSync. But i am having difficulty to understand the code. As I cant understand what is the purpose of the for loop and creating the variable "attributes" and the purpose of all those params and the prefix variable. Where as id the the primary key of my dynamo table
const AWS = require('aws-sdk');
const docClient = new AWS.DynamoDB.DocumentClient();
type Params = {
TableName: string | undefined,
Key: string | {},
ExpressionAttributeValues: any,
ExpressionAttributeNames: any,
UpdateExpression: string,
ReturnValues: string
}
async function updateTodo(todo: any) {
let params: Params = {
TableName: process.env.TODOS_TABLE,
Key: {
id: todo.id
},
ExpressionAttributeValues: {},
ExpressionAttributeNames: {},
UpdateExpression: "",
ReturnValues: "UPDATED_NEW"
};
let prefix = "set ";
let attributes = Object.keys(todo);
for (let i = 0; i < attributes.length; i++) {
let attribute = attributes[i];
if (attribute !== "id") {
params["UpdateExpression"] += prefix + "#" + attribute + " = :" + attribute;
params["ExpressionAttributeValues"][":" + attribute] = todo[attribute];
params["ExpressionAttributeNames"]["#" + attribute] = attribute;
prefix = ", ";
}
}
try {
await docClient.update(params).promise()
return todo
} catch (err) {
console.log('DynamoDB error: ', err)
return null
}
}
export default updateTodo;
This method is building the required attributes for the updateItem method.
Here's an example of what the parameters being passed to the upateItem method should look like:
const params = {
TableName: "YOUR_TABLE_NAME",
Key: {
"id": "1"
},
UpdateExpression: "set #attribute1 = :a1, #attribute2 = :a2",
ExpressionAttributeNames: {
"#attribute1": "attribute1_name"
"#attribute1": "attribute2_name"
},
ExpressionAttributeValues: {
":a1": "attribute 1 value",
":a2": "attribute 2 value"
}
};
The updateTodo method is building this parameter hash dynamically in several steps:
This block of code is creating a map called params. It's specifying the table name by looking up the TODOS_TABLE environment variable. The Key refers to the partition key, which in this example is named id.
let params: Params = {
TableName: process.env.TODOS_TABLE,
Key: {
id: todo.id
},
ExpressionAttributeValues: {},
ExpressionAttributeNames: {},
UpdateExpression: "",
ReturnValues: "UPDATED_NEW"
};
Notice how the ExpressionAttributeValues,ExpressionAttributeNames and UpdateExpression keys all have empty values.
The next step is to set the params[ExpressionAttributeValues] and params[ExpressionAttributeNames] as required by the updateItem API.
let prefix = "set ";
let attributes = Object.keys(todo);
for (let i = 0; i < attributes.length; i++) {
let attribute = attributes[i];
if (attribute !== "id") {
params["UpdateExpression"] += prefix + "#" + attribute + " = :" + attribute;
params["ExpressionAttributeValues"][":" + attribute] = todo[attribute];
params["ExpressionAttributeNames"]["#" + attribute] = attribute;
prefix = ", ";
}
}
The end result is a params map that would look something like this:
const params = {
TableName: "YOUR_TABLE_NAME",
Key: {
"id": "1"
},
UpdateExpression: "set #attribute1 = :a1, #attribute2 = :a2",
ExpressionAttributeNames: {
"#attribute1": "attribute1_name"
"#attribute1": "attribute2_name"
},
ExpressionAttributeValues: {
":a1": "attribute 1 value",
":a2": "attribute 2 value"
}
};
I'm trying to get a specific item from a table.
My DynamoDB table name is table and I have:
Name PK | Number<br/>
S: Juan | S: #####
When I try to run in Lambda I don't get any Item when it really exist one with that name... any idea why it's like that?
AWS = require("aws-sdk"),
DDB = new AWS.DynamoDB({
region: "REGION",
}),
lookup_name_str = name //From Intent variable,
params = {
TableName: "table",
KeyConditionExpression: "name = :v1",
ExpressionAttributeValues: {
":v1":{
"S": lookup_name_str
}
},
FilterExpression: 'contains(nomColaborador,:v1)',
ProjectionExpression: "Number"
};
console.log(params);
var docClient = new AWS.DynamoDB.DocumentClient();
docClient.scan(params, function(err, data){
if(err){
throw err;
}
if(data.Items && data.Items[0] && data.Items[0].Number){
console.log("There is a Name with that number");
console.log(data.Items[0]);
my_response.statusCode = 200;
my_response.body = {
"sessionAttributes": {
"extension_str": data.Items[0].Number.S,
"nomColaborador": event.currentIntent.slots.Name
},
"dialogAction":{
"type": "Close",
"fulfillmentState": "Fulfilled",
"message": {
"contentType": "PlainText",
"content": data.Items[0].Number.S
}
}
};
The main problem here is that you are doing a scan. KeyConditionExpression isn't a parameter of a scan request. If you are requesting a single item by key you want to use getItem. If you need to query data by partition key and an optional sort key you should use query.
With that all said, when you do a scan, or put a filter on a query, you really need to be sure to page through the data. You will often find that you'll get a response with no data, but a paging key to make another call.
I'm playing with the New Data API for Amazon Aurora Serverless
Is it possible to get the table column names in the response?
If for example I run the following query in a user table with the columns id, first_name, last_name, email, phone:
const sqlStatement = `
SELECT *
FROM user
WHERE id = :id
`;
const params = {
secretArn: <mySecretArn>,
resourceArn: <myResourceArn>,
database: <myDatabase>,
sql: sqlStatement,
parameters: [
{
name: "id",
value: {
"stringValue": 1
}
}
]
};
let res = await this.RDS.executeStatement(params)
console.log(res);
I'm getting a response like this one, So I need to guess which column corresponds with each value:
{
"numberOfRecordsUpdated": 0,
"records": [
[
{
"longValue": 1
},
{
"stringValue": "Nicolas"
},
{
"stringValue": "Perez"
},
{
"stringValue": "example#example.com"
},
{
"isNull": true
}
]
]
}
I would like to have a response like this one:
{
id: 1,
first_name: "Nicolas",
last_name: "Perez",
email: "example#example.com",
phone: null
}
update1
I have found an npm module that wrap Aurora Serverless Data API and simplify the development
We decided to take the current approach because we were trying to cut down on the response size and including column information with each record was redundant.
You can explicitly choose to include column metadata in the result. See the parameter: "includeResultMetadata".
https://docs.aws.amazon.com/rdsdataservice/latest/APIReference/API_ExecuteStatement.html#API_ExecuteStatement_RequestSyntax
Agree with the consensus here that there should be an out of the box way to do this from the data service API. Because there is not, here's a JavaScript function that will parse the response.
const parseDataServiceResponse = res => {
let columns = res.columnMetadata.map(c => c.name);
let data = res.records.map(r => {
let obj = {};
r.map((v, i) => {
obj[columns[i]] = Object.values(v)[0]
});
return obj
})
return data
}
I understand the pain but it looks like this is reasonable based on the fact that select statement can join multiple tables and duplicated column names may exist.
Similar to the answer above from #C.Slack but I used a combination of map and reduce to parse response from Aurora Postgres.
// declarative column names in array
const columns = ['a.id', 'u.id', 'u.username', 'g.id', 'g.name'];
// execute sql statement
const params = {
database: AWS_PROVIDER_STAGE,
resourceArn: AWS_DATABASE_CLUSTER,
secretArn: AWS_SECRET_STORE_ARN,
// includeResultMetadata: true,
sql: `
SELECT ${columns.join()} FROM accounts a
FULL OUTER JOIN users u ON u.id = a.user_id
FULL OUTER JOIN groups g ON g.id = a.group_id
WHERE u.username=:username;
`,
parameters: [
{
name: 'username',
value: {
stringValue: 'rick.cha',
},
},
],
};
const rds = new AWS.RDSDataService();
const response = await rds.executeStatement(params).promise();
// parse response into json array
const data = response.records.map((record) => {
return record.reduce((prev, val, index) => {
return { ...prev, [columns[index]]: Object.values(val)[0] };
}, {});
});
Hope this code snippet helps someone.
And here is the response
[
{
'a.id': '8bfc547c-3c42-4203-aa2a-d0ee35996e60',
'u.id': '01129aaf-736a-4e86-93a9-0ab3e08b3d11',
'u.username': 'rick.cha',
'g.id': 'ff6ebd78-a1cf-452c-91e0-ed5d0aaaa624',
'g.name': 'valentree',
},
{
'a.id': '983f2919-1b52-4544-9f58-c3de61925647',
'u.id': '01129aaf-736a-4e86-93a9-0ab3e08b3d11',
'u.username': 'rick.cha',
'g.id': '2f1858b4-1468-447f-ba94-330de76de5d1',
'g.name': 'ensightful',
},
]
Similar to the other answers, but if you are using Python/Boto3:
def parse_data_service_response(res):
columns = [column['name'] for column in res['columnMetadata']]
parsed_records = []
for record in res['records']:
parsed_record = {}
for i, cell in enumerate(record):
key = columns[i]
value = list(cell.values())[0]
parsed_record[key] = value
parsed_records.append(parsed_record)
return parsed_records
I've added to the great answer already provided by C. Slack to deal with AWS handling empty nullable character fields by giving the response { "isNull": true } in the JSON.
Here's my function to handle this by returning an empty string value - this is what I would expect anyway.
const parseRDSdata = (input) => {
let columns = input.columnMetadata.map(c => { return { name: c.name, typeName: c.typeName}; });
let parsedData = input.records.map(row => {
let response = {};
row.map((v, i) => {
//test the typeName in the column metadata, and also the keyName in the values - we need to cater for a return value of { "isNull": true } - pflangan
if ((columns[i].typeName == 'VARCHAR' || columns[i].typeName == 'CHAR') && Object.keys(v)[0] == 'isNull' && Object.values(v)[0] == true)
response[columns[i].name] = '';
else
response[columns[i].name] = Object.values(v)[0];
}
);
return response;
}
);
return parsedData;
}
I am new to AWS and dynamo. I working on my project with React.js front-end and AWS (Gateway API, Lambda, Dynamo) backend.
This is my app location:
https://www.alphaux.com
After I click "Get Hint", I receive server response. If I click on a keywords - these keywords will be added to the list of GET params for the request like:
topic=blah&keywords=blah1,blah2,blah3
Here are the details of my problem:
In my Lambda:
..
const docClient = new AWS.DynamoDB.DocumentClient({region: 'us-west-2'});
exports.handler = async (event) => { ..
from GET I am receiving the following params:
const topicName = event.queryParams.topic="React";
const keywords = event.queryParams.topic.keywords="blah1,blah2,blah3";
..some code here converts keywords string into array:
const keywordsArray = [blah1,blah2,blah3];
In dynamodb they exist in the following way:
[ { "S" : "javascript" }, { "S" : "programming" }, { "S" : "React" } ]
My Primary partition key is: id (Number)
I have tried different ways and approaches, used scan and query - nothing works. I am stuck..
I have tried the following approach:
const listToObjectMappings = () => {
let x = {};
/* keywords hardcoded for now: */
const keywords = ["javascript", "React"];
keywords.map(item => x[':' + item] = item)
return x
}
let mappings = listToObjectMappings()
let joined = Object.keys(mappings).join();
var params2 = {
TableName : "my-little-table",
FilterExpression: 'topic = :topic and #keywords IN (' + joined + ')',
ExpressionAttributeNames: {
'#keywords' : 'keywords'
},
ExpressionAttributeValues:{
":topic" : "React"
}
};
var params = {
TableName: "my-little-table",
FilterExpression: "#topic = :topic",
ExpressionAttributeNames: {
"#topic": "topic"
},
ExpressionAttributeValues: {
":topic": "React"
}
};
let result;
try {
/* scan DB */
result = await docClient.scan(params).promise();
}
catch(ex) {
result = ex;
}
When I use "scan", the result is always empty [] if I am trying to use "keywords".
It works only with bare minimum like:
const params = {
TableName : currentTable,
FilterExpression:'topic = :topic',
ExpressionAttributeValues:{
":topic" : requestedTopic
}
};
..which gives me all records based on requestedTopic.
If I am using "query" it complains that my key name ("id") is too short and has to be at least 3 chars long.
I am stuck and gracefully asking for your help!
Thank you!
If you're trying to match the topic and any one of the keywords then use something like this:
const params = {
TableName: 'mytable',
FilterExpression: '#tp = :tp AND (contains(#kw, :kw1) OR contains(#kw, :kw2))',
ExpressionAttributeNames: {
'#tp': 'topic',
'#kw': 'keywords',
},
ExpressionAttributeValues: {
':tp': 'React',
':kw1': 'react',
':kw2': 'react-router',
},
};
If you're trying to match the topic and all of the keywords then use something like this:
const params = {
TableName: 'mytable',
FilterExpression: '#tp = :tp AND contains(#kw, :kw1) AND contains(#kw, :kw2)',
ExpressionAttributeNames: {
'#tp': 'topic',
'#kw': 'keywords',
},
ExpressionAttributeValues: {
':tp': 'React',
':kw1': 'react',
':kw2': 'react-router',
},
};