I want to do conditional putItem call into DynamoDb i.e. don't insert an entry in dynamoDb if the primaryKey(partitionKey + Sort Key already exists). My schema's key look like this:
PartitionKey: abc:def
SortKey:abc:123
To do a conditional I do something like this:
private static final String PK_DOES_NOT_EXIST_EXPR = "attribute_not_exists(%s)";
final String condition = String.format(PK_DOES_NOT_EXIST_EXPR,
record.getPKey() + record.getSortKey);
final PutItemEnhancedRequest putItemEnhancedRequest = PutItemEnhancedRequest
.builder(Record.class)
.conditionExpression(
Expression.builder()
.expression(condition)
.build()
)
.item(newRecord)
.build();
However I run into following error
Exception in thread "main" software.amazon.awssdk.services.dynamodb.model.DynamoDbException: Invalid ConditionExpression: Syntax error; token: ":123", near: "abc:123)" (Service: DynamoDb, Status Code: 400
I am assuming this is because of : present in my condition, because the same expression without : in the key succeeds. Is there a way to fix this?
Your condition should include the name of the partition key attribute, not its value. For example:
attribute_not_exists(pk)
Also, see Uniqueness for composite primary keys for an explanation of why you only need to indicate the partition key attribute name, not both the partition key and the sort key attribute names. So, the following, while not harmful, is unnecessary:
attribute_not_exists(pk) AND attribute_not_exists(sk)
Related
AWS DynamoDB table has :
Client (Primary Key),
folder_location (non-key attribute),
script_name (non-key attribute)
I want to retrieve records using Client and folder_location attributes using BatchGetItemRequest.
But getting below error:
Failed to retrieve items.com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException: The provided key element does not match the schema (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException;
Is there a way to do with BatchGetItemRequest only ?
You could instead use a Query or Scan operation and specify a Filter Expression to limit results based on the folder_location.
To use BatchGetItemRequest you must provide the key, per https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_BatchGetItem.html:
The BatchGetItem operation returns the attributes of one or more items
from one or more tables. You identify requested items by primary key.
Keys - An array of primary key attribute values that define specific
items in the table. For each primary key, you must provide all of the
key attributes. For example, with a simple primary key, you only need
to provide the partition key value. For a composite key, you must
provide both the partition key value and the sort key value.
I recommend you double check that the key you are providing matches the key defined in the table (name and data type). If so, then try one of those other options with a FilterExpression.
I have Tags as partition key in my table, and when I am trying to query I am getting AttributeError.
Below is my code:
kb_table = boto3.resource('dynamodb').Table('table_name')
result = kb_table.query(
KeyConditionExpression=Key('Tags').contains('search term')
)
return result['Items']
Error:
"errorMessage": "'Key' object has no attribute 'contains'"
Basically I want to search through the table where I the field is having that search term. I have achived it using scan but I have read everywhere that we should not use that.
result = kb_table.scan(
FilterExpression="contains (Tags, :titleVal)",
ExpressionAttributeValues={ ":titleVal": "search term" }
)
So I have changed my partition-key to Tags along with a sort-key so that I can achieve this using query but now I am getting this error.
Any idea how to get this working?
In order to use Query you must specify one partition to access, you cannot wildcard a partition or specify multiple keys.
KeyConditionExpression
The condition that specifies the key value(s)
for items to be retrieved by the Query action.
The condition must perform an equality test on a single partition key
value.
Assuming you want to search the whole table for tags, a scan is the most appropriate approach.
EDIT: You can use Query with the exact search term, but im guessing that is not what you want.
kb_table = boto3.resource('dynamodb').Table('table_name')
result = kb_table.query(
KeyConditionExpression=Key('Tags').eq('search term')
)
return result['Items']
I am trying to query a dynamodb table. When I'm using the begins with operator I'm getting the following error.
{u'Message': u'All queries must have a condition on the hash key, and
it must be of type EQ', u'__type':
u'com.amazon.coral.validate#ValidationException'}
result_set = tb_places.query_2(
place_name__beginswith="ame",
)
Here place_name is a Global Secondary Index
Regardless if you are querying a table or index, the only operator that can be applied to a Hash Key attribute is the EQ. Alternatively, you could use BEGINS_WITH on a Range Key.
For a query on a table, you can have conditions only on the table
primary key attributes. You must provide the hash key attribute name
and value as an EQ condition. You can optionally provide a second
condition, referring to the range key attribute.[...]
For a query on an index, you can have conditions only on the index key
attributes. You must provide the index hash attribute name and value
as an EQ condition. You can optionally provide a second condition,
referring to the index key range attribute.
Source: http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html
I am trying to use the Java AWS sdk to get a document based on a Global Secondary Index.
Setup as follows:
Hash Key: MyId - Number
Range Key: MyDate - String
Here is my code to:
Map<String, AttributeValue> key = new HashMap<String, AttributeValue>();
key.put("MyId", new AttributeValue().withN("1234"));
key.put("MyDate", new AttributeValue().withS("2014-10-12"));
GetItemRequest go = new GetItemRequest().withTableName(tableName).withKey(key);
GetItemResult result = getDynamoDBClient().getItem(gi);
But this always returns :
The provided key element does not match the schema (Service:
AmazonDynamoDBv2; Status Code: 400
What am I dong wrong?
A few notes, first you talk about GSI but you are doing GetItemRequest by primary key. So perhaps you're missing something in your question.
Did you write in your question the primary key of the table or the GSI definition?
You can only Query by GSI, Get is still based on primary key.
I am using DynamoDB to store events.
They are stored in 1 event table with a hash key 'Source ID' and a range key 'version'. Every time a new event occurs for a source, i want to add a new item with the source ID and an increased version nr.
Is it possible to specify a conditional write so that a duplicate item (same hash key and same range key) can never exist? And if so, how would you do this?
I have done this successfully for tables with just a Hash Key:
Map<String, ExpectedAttributeValue> expected = new HashMap<String, ExpectedAttributeValue>();
expected.put("key", new ExpectedAttributeValue().withExists(false));
But not sure how to handle hash + range keys....
I don't know Java SDK well but you can specify "Exist=False" on both the range_key and the hash_key.
Maybe a better idea could be to use a timestamp instead of a version number ? Otherwise, there are also techniques to generate unique ids.
I was trying to enforce a unique combination of hash and range keys and came across this post. I found that it didn't completely answer my question but certainly pointed me in the right direction. This is an attempt to tidy up the loose ends.
It seems that DynamoDB actually enforces a unique combination of hash and range key by design. I quote
"All items in the table must have a value for the primary key attribute and Amazon DynamoDB ensures that the value for that name is unique"
from http://aws.amazon.com/dynamodb/ under the section with the heading Primary Key.
In my own tests using putItem with the aws-sdk for nodejs I was able to post two identical items without generating an error. When I checked the database, only one item was actually inserted. It seems that the second call to putItem with the same hash and range key combination is treated like an update to the original item.
I too received the error "Cannot expect an attribute to have a specified value while expecting it to not exist" when I tried to set the exist=false option on the hash key and range key with the values set. To resolve this error, I removed the value under the expected hash and range key and it started to generate a validation error when I tried to insert the same key twice.
So, my insert command looks like this (will be different for Java, but hopefully you get the idea)
{ "TableName": "MyTableName",
"Item" : {
"HashKeyFieldName": {
"S": HashKeyValue
},
"RangeKeyFieldName": {
"N": currentTime.getTime().toString()
},
"OtherField": {
"N": "61404032632"
}
},
"Expected": {
"HashKeyFieldName" : { "Exists" : false},
"RangeKeyFieldName" : { "Exists" : false}
}
}
Whereas originally I was trying to do a conditional insert to check if there was a hash value and range value the same as what I was trying to insert, now I just need to check if the HashField and RangeField exist at all. If they exist, that means I am updating an item rather than inserting.