I need to execute query like this:
select * from table where sampling_date like "2020-05-%"
To do this, I'm calling for
db.query({
TableName: "Tubes",
Select: "ALL_ATTRIBUTES",
IndexName: "sampling_date_idx",
KeyConditionExpression: " sampling_date > :sampling_date ",
ExpressionAttributeValues:{ ':sampling_date': {'S': '2020-05-'}}
}, function(error: AWSError, data: QueryOutput){
console.log(error, data);
})
And I get this error message:
{"errorType":"Error","errorMessage":"{\"message\":\"Query key condition not supported\",\"code\":\"ValidationException\",
My table:
this.tubes = new dynamodb.Table(this, "tubes", {
tableName: "Tubes",
billingMode: dynamodb.BillingMode.PAY_PER_REQUEST,
partitionKey: {
name: "id",
type: dynamodb.AttributeType.STRING
},
pointInTimeRecovery: true,
removalPolicy: cdk.RemovalPolicy.RETAIN
});
this.tubes.addGlobalSecondaryIndex({
indexName: "sampling_date_idx",
sortKey: {
name: 'sampling_date_srt',
type: AttributeType.STRING
},
partitionKey: {
name: "sampling_date",
type: AttributeType.STRING,
},
})
I think there are two issues in your current code -
In KeyConditionExpression, there must be an equality condition on a single partition key value. In your case, it must include "sampling_date = :sampling_date".
Please read "KeyConditionExpression" section in -
https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html
In short, you only can perform equality test against partition key.
I am not sure which language you use. I suspect your syntax for ExpressionAttributeValues is not correct.
The syntax given in AWS doc is -
"ExpressionAttributeValues": {
"string" : {
"B": blob,
"BOOL": boolean,
"BS": [ blob ],
"L": [
"AttributeValue"
],
"M": {
"string" : "AttributeValue"
},
"N": "string",
"NS": [ "string" ],
"NULL": boolean,
"S": "string",
"SS": [ "string" ]
}
}
In your case, it may be something like -
"ExpressionAttributeValues": {
":sampling_date": {"S": "2020-05-01"}
}
My experience is in C#, it may be something like -
ExpressionAttributeValues = new Dictionary<string, AttributeValue>()
{
{ ":sampling_date", new AttributeValue{S = "2005-05-01"} }
}
To solve your problem, you may need to use another attribute as the index's partition key. sampling_date only can be used as sort key.
sampling_date is the partition key for your GSI sampling_date_idx.
DynamoDB documentation says that in key condition expressions:
You must specify the partition key name and value as an equality condition.
Source: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html#Query.KeyConditionExpressions
So sampling_date can only be used with the "equal to" comparison operator. None of the other operators like less than, greater than, between, contains, begins with, etc. can be used with sampling_date.
However, these operators can be used with a sort key!
So if you can redesign your table and/or indexes such that sampling_date becomes a sort key of some index, you can use begins_with on it.
Here's a suggestion:
Create a GSI with partition key = sampling_year & sort key = sampling_date.
Then if your table has the following items:
{
"id": "id1",
"sampling_year": 2020,
"sampling_date": "2020-04-01"
}
{
"id": "id2",
"sampling_year": 2020,
"sampling_date": "2020-05-01"
}
{
"id": "id3",
"sampling_year": 2020,
"sampling_date": "2020-06-01"
}
And you use the following Node.js code:
let AWS = require("aws-sdk")
let dc = new AWS.DynamoDB.DocumentClient()
dc.query({
TableName: "Tubes",
IndexName: "sampling_year-sampling_date-index",
KeyConditions: {
"sampling_year": {
ComparisonOperator: "EQ",
AttributeValueList: [2020]
},
"sampling_date": {
ComparisonOperator: "BEGINS_WITH",
AttributeValueList: ["2020-05-"]
}
}
}
You'll get your desired output:
{
"id": "id2",
"sampling_year": 2020,
"sampling_date": "2020-05-01"
}
Try
KeyConditionExpression: `begins_with(sampling_date, :sampling_date)`
See available condition expressions here...
https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Condition.html
Related
In my dynamodb table, I have a scenario where when I add a new item to my table, I need to also increment a counter for another entry in my table.
For instance, when USER#1 follows USER#2, I would like to increment followers count for USER#2.
HashKey
RangeKey
counter
USER1
USER2
USER3
USER2
USER2
USER2
2
I do not want to use auto-increment as I want to control how the increment happens.
Naturally, everything works as expected if I make two update calls to dynamodb. One to create the relationship between users and another to update the count for the other user.
The question is, if it is a good approach to make two such calls or would a transactWrite be a better alternative.
If so how could I make an increment using transactwrite api.
I can add items using the following approach. But I am not sure how I can increment
"TransactItems": [
{
"Put": {
"TableName": "Table",
"Item": {
"hashKey": {"S":"USER1"},
"rangeKey": {"S":"USER2"}
}
}
},
{
"Update": {
"TableName": "TABLE",
"Key": {
"hashKey": {"S":"USER2"},
"rangeKey": {"S":"USER2"}
},
"ConditionExpression": "#cefe0 = :cefe0",
"ExpressionAttributeNames": {"#cefe0":"counter"},
"ExpressionAttributeValues": ?? how do I increment here
}
}
]
Transactions would defintely be the best way to approach it, you can increment using SET in the UpdateExpression:
"TransactItems": [
{
"Put": {
"TableName": "Table",
"Item": {
"hashKey": {"S":"USER1"},
"rangeKey": {"S":"USER2"}
}
}
},
{
"Update": {
"TableName": "TABLE",
"Key": {
"hashKey": {"S":"USER2"},
"rangeKey": {"S":"USER2"}
},
"UpdateExpression": "SET #cefe0 = #cefe0 + :cefe0",
"ExpressionAttributeNames": {"#cefe0":"counter"},
"ExpressionAttributeValues": {"cefe0": {"N": "1"}}
}
}
]
I am able to SET values for reserved keywords but not able to remove certain keywords in dynamodb.
My expression is below. Not able to remove data if I give like this. But able to remove size though.
Invalid UpdateExpression: Attribute name is a reserved keyword; reserved keyword:data
{
ConditionExpression: "#id = :id",
ExpressionAttributeNames: {
"#id": "id",
“#name": “name",
},
ExpressionAttributeValues: {
":id": “1234566",
“:name": “John",
},
Key: {
id: "1234566",
},
ReturnValues: "ALL_NEW",
TableName: “table_name",
UpdateExpression: "SET #name = :name REMOVE size, data",
}
Modified to include data in expression attribute names as below. Still throws error.
{
ConditionExpression: "#id = :id",
ExpressionAttributeNames: {
"#id": "id",
“#name": “name",
“#size": “size",
“#data": “data",
},
ExpressionAttributeValues: {
":id": “1234566",
“:name": “John",
“:size": undefined,
“:data": undefined,
},
Key: {
id: "1234566",
},
ReturnValues: "ALL_NEW",
TableName: "table_name",
UpdateExpression: "SET #name = :name, #size = :size, #data =
:data REMOVE size, data”,
}
Remove size and data from ExpressionAttributeValues and from SET UpdateExpression. Add #size and #data to REMOVE on UpdateExpression instead of size and data.
{
ConditionExpression: "#id = :id",
ExpressionAttributeNames: {
"#id": "id",
“#name": “name",
“#size": “size",
“#data": “data",
},
ExpressionAttributeValues: {
":id": “1234566",
“:name": “John"
},
Key: {
id: "1234566",
},
ReturnValues: "ALL_NEW",
TableName: "table_name",
UpdateExpression: "SET #name = :name, REMOVE #size, #data”,
}
Reserved words in DynamoDB
to do that dynamically you need only to provide a json object with this function
it's create dynamically
-ExpressionAttributeNames
-ExpressionAttributeValues
-UpdateExpression
also skip reserved keyword for dynamodb
def generate_update_att(body):
"""
#############input
body={
'newval':newData['newval'],
'lastname':newData['lastname'],
'name':newData['name']
}
#################output
-ExpressionAttributeNames
-ExpressionAttributeValues
-UpdateExpression
"""
UpdateExpression = ["set "]
ExpressionAttributeValues = dict()
ExpressionAttributeNames=dict()
for key, val in body.items():
UpdateExpression.append(f" #{key} = :{key},")
ExpressionAttributeValues[f":{key}"] = val
ExpressionAttributeNames[f"#{key}"] = key
return "".join(UpdateExpression)[:-1], ExpressionAttributeValues,ExpressionAttributeNames
function Call
my_data={
'newval':newData['newval'],
'lastname':newData['lastname'],
'name':newData['name']
}
a, v,z = generate_update_att(my_data)
my_table.update_item(
Key={'id':my_id},
UpdateExpression=my_UpdateExpression,
ExpressionAttributeValues=dict(my_ExpressionAttributeValues),
ExpressionAttributeNames=dict(my_ExpressionAttributeNames)
)
I would like to fetch data from DynamoDB database by using [AWS-DynamoDB-Data-Mapper][1]. I am using QueryIterator and to provide indexName, limit, filter purpose, I am using [QueryOptions][2]. In QueryOptions, I would like to use filter?: ConditionExpression. Where I would like to use contains() function, but I am not able to form ConditionExpression in the below example -
{
"Id": {"N": "456" },
"ProductCategory": {"S": "Sporting Goods" },
"Price": {"N": "650" }
}
const paginator = dbMapper.query(
Products,
{
Id: 456,
Price: between(500, 1000)
},
{
indexName: 'indexByPriceId',
limit: 10,
scanIndexForward: false,
filter: contains(ProductCategory, 'goods')
}
);
It returns an syntax error in this line filter: contains(ProductCategory, 'goods'), it says ContainsPredicate is not a ConditionExpression.
Please help me out by providing me the ConditionExpression in QueryOptions.
[1]: https://github.com/awslabs/dynamodb-data-mapper-js
[2]: https://awslabs.github.io/dynamodb-data-mapper-js/packages/dynamodb-data-mapper/interfaces/queryoptions.html#projection
I'm trying to get a specific item from a table.
My DynamoDB table name is table and I have:
Name PK | Number<br/>
S: Juan | S: #####
When I try to run in Lambda I don't get any Item when it really exist one with that name... any idea why it's like that?
AWS = require("aws-sdk"),
DDB = new AWS.DynamoDB({
region: "REGION",
}),
lookup_name_str = name //From Intent variable,
params = {
TableName: "table",
KeyConditionExpression: "name = :v1",
ExpressionAttributeValues: {
":v1":{
"S": lookup_name_str
}
},
FilterExpression: 'contains(nomColaborador,:v1)',
ProjectionExpression: "Number"
};
console.log(params);
var docClient = new AWS.DynamoDB.DocumentClient();
docClient.scan(params, function(err, data){
if(err){
throw err;
}
if(data.Items && data.Items[0] && data.Items[0].Number){
console.log("There is a Name with that number");
console.log(data.Items[0]);
my_response.statusCode = 200;
my_response.body = {
"sessionAttributes": {
"extension_str": data.Items[0].Number.S,
"nomColaborador": event.currentIntent.slots.Name
},
"dialogAction":{
"type": "Close",
"fulfillmentState": "Fulfilled",
"message": {
"contentType": "PlainText",
"content": data.Items[0].Number.S
}
}
};
The main problem here is that you are doing a scan. KeyConditionExpression isn't a parameter of a scan request. If you are requesting a single item by key you want to use getItem. If you need to query data by partition key and an optional sort key you should use query.
With that all said, when you do a scan, or put a filter on a query, you really need to be sure to page through the data. You will often find that you'll get a response with no data, but a paging key to make another call.
I need another set of eyes on this. For the life of me I see no issues with this parameter set, used for Dynamo DocumentClient, update method - (here: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB/DocumentClient.html#update-property ).
{
TableName: "mygame-dev",
Key: { pk: "09d017aa-cbf7-42ce-be6a-a94ecb58f9a7", sk: "GAME" },
ExpressionAttributeNames: { "#GAMELASTUPDATED": "gameLastUpdated", "#GAMETITLE": "gameTitle" },
ExpressionAttributeValues: { ":gamelastupdated": 1556376010704, ":gametitle": "test title 1" },
UpdateExpression: "SET #GAMELASTUPDATED = :gamelastupdated, #GAMETITLE = :gametitle",
ReturnValues: "ALL_NEW"
};
Error:
ValidationException: ExpressionAttributeNames can only be specified
when using expressions
Any thoughts?
Disregard, this was a copy and paste issue... I was using "query" not "update"
Was:
const updateGameResult = await ddbCall("query", params);
Should have been:
const updateGameResult = await ddbCall("update", params);