I am working on an application that uses DynamoDB.
Is there a way where I can create a GSI with multiple attributes. My aim is to query the table with a query of following kind:
(attrA.val1 === someVal1 AND attrB.val2 === someVal2 AND attrC.val3 === someVal3)
OR (attrA.val4 === someVal4 AND attrB.val5 === someVal5 AND attrC.val6 === someVal6)
I am aware we can use Query when we have the Key Attribute and when Key Attribute is unknown we can use Scan operations. I am also aware of GSI if we need to query with non-key attributes. But I need some help in this scenario. Is there a way to model GSI to suit the above query.
I have the below item (i.e. data) on my Movies tables. The below query params works fine for me.
You can add the third attribute as present in the OP. It should work fine.
DynamoDB does support the complex condition on FilterExpression.
Query table based on some condition:-
var table = "Movies";
var year_val = 1991;
var title = "Movie with map attribute";
var params = {
TableName : table,
KeyConditionExpression : 'yearkey = :hkey and title = :rkey',
FilterExpression : '(records.K1 = :k1Val AND records.K2 = :k2Val) OR (records.K3 = :k3Val AND records.K4 = :k4Val)',
ExpressionAttributeValues : {
':hkey' : year_val,
':rkey' : title,
':k3Val' : 'V3',
':k4Val' : 'V4',
':k1Val' : 'V1',
':k2Val' : 'V2'
}
};
docClient.query(params, function(err, data) {
if (err) {
console.error("Unable to read item. Error JSON:", JSON.stringify(err,
null, 2));
} else {
console.log("GetItem succeeded:", JSON.stringify(data, null, 2));
}
});
My Data:-
Result:-
GetItem succeeded: {
"Items": [
{
"title": "Movie with map attribute",
"yearkey": 1991,
"records": {
"K3": "V3",
"K4": "V4",
"K1": "V1",
"K2": "V2"
}
}
],
"Count": 1,
"ScannedCount": 1
}
Related
I am using JS SDK with DynamoDB to fetch data.
I am able to fetch data from my table using simple query with partition key and sort key.
My sort key sk has records -
Year#Batch#Rate
If I pass var sk = "2006#CSE#90"; it returns all of records matching this,
Requirement - How can I get all products with year 2006 , Batch CSE AND Rate =>90
readItem_pbro(){
console.log("inside pbro");
var table2 = "pbro";
var pk = "1";
var sk = "2006#CSE#90";
var params2 = {
TableName: table2,
Key:{
"pk": pk,
"sk": sk
}
};
Edit 1 :: Created a different column for score/rate as score. It is numeric.
Now my query in JS is -
but I am getting error - ValidationException: The provided key element does not match the schema
readItem_score_filter(){
console.log("inside pbro");
var table2 = "pbro";
var pk = "1"; // string
var sk = "2006#CSE"; // string
var score = 90; //number
var params2 = {
TableName: table2,
Key:{
"pk": pk,
"sk": sk,
FilterExpression:'score >=:score',
}
};
what is wrong in my FilterExpression.
Edit 2 :: Added Key condition Expression but issue still remains the same
Error: ValidationException: The provided key element does not match the schema
Here is my complete function now:
readItem_score_filter(){
console.log("inside pbro");
var table2 = "pbro";
var pk = "1"; //string
var sk = "2006#CSE"; // string
var score = 90; //number
var params2 = {
TableName: table2,
Key:{
"pk": pk,
"sk": sk,
"score": score,
KeyConditionExpression: 'pk = :pk AND sk=:sk',
FilterExpression: "score >=:score",
}
};
this.user.docClient.get(params2, function(err, data) {
if (err) {
console.log(err);
} else {
console.log(data);
}
});
}
Screenshot of table attached incase you need to see::
If "2006#CSE#90" this is the value of sort key column then you cant do anything at Dynamodb level..
comparings like this can be done through regular expressions but DynamoDB doesn't support Regular Expressions.
you simply need to get results and then seperate these values and compare ..
Updated :- use different column for score.
And use Filter Expression to get records having rate more than 90.
I dont know python , but still am trying here
var params2 = {
TableName: "pbro",
KeyConditionExpression: "pk = :pk AND sk =:sk",
FilterExpression: "score >= :score"
};
I'm trying to get a specific item from a table.
My DynamoDB table name is table and I have:
Name PK | Number<br/>
S: Juan | S: #####
When I try to run in Lambda I don't get any Item when it really exist one with that name... any idea why it's like that?
AWS = require("aws-sdk"),
DDB = new AWS.DynamoDB({
region: "REGION",
}),
lookup_name_str = name //From Intent variable,
params = {
TableName: "table",
KeyConditionExpression: "name = :v1",
ExpressionAttributeValues: {
":v1":{
"S": lookup_name_str
}
},
FilterExpression: 'contains(nomColaborador,:v1)',
ProjectionExpression: "Number"
};
console.log(params);
var docClient = new AWS.DynamoDB.DocumentClient();
docClient.scan(params, function(err, data){
if(err){
throw err;
}
if(data.Items && data.Items[0] && data.Items[0].Number){
console.log("There is a Name with that number");
console.log(data.Items[0]);
my_response.statusCode = 200;
my_response.body = {
"sessionAttributes": {
"extension_str": data.Items[0].Number.S,
"nomColaborador": event.currentIntent.slots.Name
},
"dialogAction":{
"type": "Close",
"fulfillmentState": "Fulfilled",
"message": {
"contentType": "PlainText",
"content": data.Items[0].Number.S
}
}
};
The main problem here is that you are doing a scan. KeyConditionExpression isn't a parameter of a scan request. If you are requesting a single item by key you want to use getItem. If you need to query data by partition key and an optional sort key you should use query.
With that all said, when you do a scan, or put a filter on a query, you really need to be sure to page through the data. You will often find that you'll get a response with no data, but a paging key to make another call.
Say I want to add a record like below:
{
id:876876,
username:example,
email:xxxxxx#xxx.com,
phone:xxxxxxx,
created_at:current_date,
updated_at:current_date
}
What I want is to check with the id if record exist created_date should not be modified only updated_date should have current_date.
Can we do this with put method without having to be making a get item call?
To create a new item or update an existing item with conditional expression. Prefer to use updateItem rather than putItem.
If the Hash key (Primary key) doesn't exist both putItem and updateItem create a new record. If the record already exist, putItem will completely override the existing row but updateItem only updates the attributes passed in UpdateExpression not a whole record.
In your case, use if_not_exists() function to check whether the created_at field already exists or not. If exists created_at will not be overridden.
Update expression: "SET #email = :email, #created_at = if_not_exists(#created_at, :created_at), #updated_at = :updated_at"
Sample snippet
var params = {
ExpressionAttributeNames: {
"#email": "email",
"#created_at": "created_at",
"#updated_at": "updated_at"
},
ExpressionAttributeValues: {
":email": {
S: "test2#grr.la"
},
":created_at": {
S: date.toISOString()
},
":updated_at": {
S: date.toISOString()
}
},
Key: {
"id": {
S: "T1"
}
},
ReturnValues: "ALL_NEW",
TableName: "stack",
UpdateExpression: "SET #email = :email, #created_at = if_not_exists(#created_at, :created_at), #updated_at = :updated_at"
};
ddb.updateItem(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data);
})
I am new to AWS and dynamo. I working on my project with React.js front-end and AWS (Gateway API, Lambda, Dynamo) backend.
This is my app location:
https://www.alphaux.com
After I click "Get Hint", I receive server response. If I click on a keywords - these keywords will be added to the list of GET params for the request like:
topic=blah&keywords=blah1,blah2,blah3
Here are the details of my problem:
In my Lambda:
..
const docClient = new AWS.DynamoDB.DocumentClient({region: 'us-west-2'});
exports.handler = async (event) => { ..
from GET I am receiving the following params:
const topicName = event.queryParams.topic="React";
const keywords = event.queryParams.topic.keywords="blah1,blah2,blah3";
..some code here converts keywords string into array:
const keywordsArray = [blah1,blah2,blah3];
In dynamodb they exist in the following way:
[ { "S" : "javascript" }, { "S" : "programming" }, { "S" : "React" } ]
My Primary partition key is: id (Number)
I have tried different ways and approaches, used scan and query - nothing works. I am stuck..
I have tried the following approach:
const listToObjectMappings = () => {
let x = {};
/* keywords hardcoded for now: */
const keywords = ["javascript", "React"];
keywords.map(item => x[':' + item] = item)
return x
}
let mappings = listToObjectMappings()
let joined = Object.keys(mappings).join();
var params2 = {
TableName : "my-little-table",
FilterExpression: 'topic = :topic and #keywords IN (' + joined + ')',
ExpressionAttributeNames: {
'#keywords' : 'keywords'
},
ExpressionAttributeValues:{
":topic" : "React"
}
};
var params = {
TableName: "my-little-table",
FilterExpression: "#topic = :topic",
ExpressionAttributeNames: {
"#topic": "topic"
},
ExpressionAttributeValues: {
":topic": "React"
}
};
let result;
try {
/* scan DB */
result = await docClient.scan(params).promise();
}
catch(ex) {
result = ex;
}
When I use "scan", the result is always empty [] if I am trying to use "keywords".
It works only with bare minimum like:
const params = {
TableName : currentTable,
FilterExpression:'topic = :topic',
ExpressionAttributeValues:{
":topic" : requestedTopic
}
};
..which gives me all records based on requestedTopic.
If I am using "query" it complains that my key name ("id") is too short and has to be at least 3 chars long.
I am stuck and gracefully asking for your help!
Thank you!
If you're trying to match the topic and any one of the keywords then use something like this:
const params = {
TableName: 'mytable',
FilterExpression: '#tp = :tp AND (contains(#kw, :kw1) OR contains(#kw, :kw2))',
ExpressionAttributeNames: {
'#tp': 'topic',
'#kw': 'keywords',
},
ExpressionAttributeValues: {
':tp': 'React',
':kw1': 'react',
':kw2': 'react-router',
},
};
If you're trying to match the topic and all of the keywords then use something like this:
const params = {
TableName: 'mytable',
FilterExpression: '#tp = :tp AND contains(#kw, :kw1) AND contains(#kw, :kw2)',
ExpressionAttributeNames: {
'#tp': 'topic',
'#kw': 'keywords',
},
ExpressionAttributeValues: {
':tp': 'React',
':kw1': 'react',
':kw2': 'react-router',
},
};
I am little confused about retrieving data from dynamodb ... connecting is not an issue cause I am getting The provided key element does not match the schema
:the example provided from AWS
var table = new AWS.DynamoDB({params: {TableName: 'MY_TABLE'}});
var key = 'UNIQUE_KEY_ID';
var itemParams = {Item: {id: {S: key}, data: {S: 'data'}}};
table.getItem({Key: {id: {S: key}}}, function(err, data) {
console.log(data.Item); // print the item data
});
in my case the unique key is "time" and what I want to do is retrieve by key (not unique)
getItem only works on the primary key. From the docs: The GetItem operation returns a set of attributes for the item with the given primary key. See docs.
To solve this, create a Global Secondary Index that has "key" as HASH and "time" as RANGE. Then do a query operation using that index as IndexName:
var params = {
IndexName: 'your-new-GSI-index',
KeyConditionExpression: '#key = :key',
ExpressionAttributeNames: { '#key': 'key },
ExpressionAttributeValues: { ':key': { S: yourKeyVar } }
}
table.query(params, callback);
(did not test this code, but should work)