I want to create a DynamoDB table which contains only one order entity with following attributes:
OrderId
OrderStatus
Price
Access patterns I try to achieve:
Get single order by id
Filter orders by status
Design in my mind for above access patterns is to create simple primary key consisting of OrderId and one local secondary index with composite primary key consisting of OrderId and OrderStatus. AWS CLI command for achieving this:
aws dynamodb create-table \
--table-name Order \
--attribute-definitions \
AttributeName=OrderId,AttributeType=S \
AttributeName=OrderStatus,AttributeType=S \
--key-schema \
AttributeName=OrderId,KeyType=HASH \
--local-secondary-indexes \
"[{\"IndexName\": \"OrderStatusIndex\",
\"KeySchema\":[{\"AttributeName\": \"OrderId\",\"KeyType\":\"HASH\"},
{\"AttributeName\":\"OrderStatus\",\"KeyType\":\"RANGE\"}],
\"Projection\":{\"ProjectionType\":\"INCLUDE\", \"NonKeyAttributes\":[\"Price\"]}}]" \
--provisioned-throughput \
ReadCapacityUnits=1,WriteCapacityUnits=1 \
--table-class STANDARD
When running this command, I get:
An error occurred (ValidationException) when calling the CreateTable
operation: One or more parameter values were invalid: Table KeySchema
does not have a range key, which is required when specifying a
LocalSecondaryIndex
How can get rid of this error while sticking to my design? Or if it is not possible how should I design my table in this case?
The definition of a Local Secondary index is that it has the same Partition Key as the table , but a difference range key.
In order to have a partition key other than what the table uses, you'll need a Global Secondary Index.
Related
i am having issues getting particular delete-item request to work through the AWS CLI.
Here is what I am trying to do:
Perform a scan operation on my DynamoDB table to return all results that match a filter expression on a field that ISN'T the partition key
For all items that match this query, delete them from the table
Here is the command I am trying to run..
aws dynamodb scan \
--filter-expression "EnvironmentGroup = :EnvironmentGroup" \
--expression-attribute-values '{":EnvironmentGroup":{"S":"deleteThisGroup"}}' \
--table-name "MyTable"
--query "Items[*]" \
# use jq to get each item on its own line
| jq --compact-output '.[]' \
# replace newlines with null terminated so
# we can tell xargs to ignore special characters
| tr '\n' '\0' \
| xargs -0 -t -I keyItem \
# use the whole item as the key to delete (dynamo keys *are* dynamo items)
aws dynamodb delete-item --table-name "MyTable" --key=keyItem
So in the above example, I want to perform a Scan on the MyTable table and return all items that have the EnvironmentGroup field set to deleteThisGroup. I then want each of these items to be deleted from the table.
This isn't working for me. If I take out the delete-item line, the command works and returns a list of all the items, but if I add the delete-item back in, I get Error parsing parameter '--key': Expected: '=', received: 'EOF' for input: keyItem
What am I doing wrong here?
delete-item is to delete one item
If you want to delete multiple items, use batch-write-item
I wanted to run a greater than Query against the primary key of my table. Later I came to know that greater than queries can only be executed on sort keys, and not on primary keys. So, I have now re-designed my table, and here's a screenshot of the new it : (StoreID is the Primary key, & OrderID is the Sort key)
How should I format the Query, if I want to run a query like return those items whose 'OrderID' > 1005?
More particularly, what should I mention in the Query condition to meet my requirements?
Thanks a lot!
You can use the following CLI command to run query "return those items in store with storeid='STR100' whose 'OrderID' > 1005".
aws dynamodb query --table-name <table-name> --key-condition-expression "StoreID = :v1 AND OrderID > :v2" --expression-attribute-values '{":v1": {"S": "STR100"}, ":v2": {"N": 1005}}'
I need to design a DynamoDB schema to store items whose attributes are:
tid: a UUID string which is a unique identifier of the item
timestamp: an ISO-8601-formatted string representing a date and time related to the item
Other stuff...
and for the following query (really want to avoid having to do any scans) patterns:
Query by tid
Query by exact timestamp, and by relational ordering expressions (e.g., <=, BETWEEN, etc.) on timestamp. I.e., query all items from a certain date-time range without knowing their tids in advance.
Is this possible to do efficiently in DynamoDB, or is there perhaps another AWS solution that would serve me better?
Given a DynamoDB table as follows:
partition key: tid, type string
sort key: timestamp, type string
You can query on:
tid = 5
tid = 5, timestamp between 2018-12-21T09:00:00Z and 2018-12-21T15:00:00Z
Try it out using the awscli, for example to query all items with tid=5:
aws dynamodb query \
--table-name mytable \
--key-condition-expression "tid = :tid" \
--expression-attribute-values '{":tid":{"S":"5"}}'
To query all items for tid=5 and timestamp between 09:00 and 15:00 on 2015-12-21:
aws dynamodb query \
--table-name mytable \
--key-condition-expression "tid = :tid AND #ts BETWEEN :ts1 AND :ts2" \
--expression-attribute-values '{":tid":{"S":"5"}, ":ts1":{"S":"2015-12-21T09:00:00Z"}, ":ts2":{"S":"2015-12-21T15:00:00Z"}}' \
--expression-attribute-names '{"#ts":"timestamp"}'
Note: because timestamp is a reserved keyword in DynamoDB, you have to escape it using the expression attribute names.
You could also create the timestamp attribute as a number and then store epoch times, if you prefer.
To query all items with timestamp between 09:00 and 15:00 on 2015-12-21, regardless of tid, cannot be done with the same partition/sort key schema. You would need to add a Global Secondary Index something like this:
GSI partition key: yyyymmdd, type string
GSI sort key: timestamp, type string
Now you can query for items with a given timestamp range, as long as they're on the same day (they have the same YYYYMMDD, which might be a reasonable restriction). Or you could go to YYYYMM as the partition key allowing a wider timestamp range. At this point you really need to understand the use cases for queries to decide if YYYYMMDD (restricting queries to a single day) is right. See How to query DynamoDB by date with no obvious hash key for more on this idea.
I'm trying to use the query expression an getting
ATTEMPT EDIT - 1 PASS
An error occurred (ValidationException) when calling the Query operation: Invalid KeyConditionExpression: Syntax error; token: "{", near: "{""
This query looks like
aws dynamodb query --table-name name-of-table --key-condition-expression 'id=:91'
I've tried '"id"="91"', ':id=":91"', etc.
id is the partition key so this is also the required attribute needed.
Try this
aws dynamodb query --table-name name-of-table --key-condition-expression 'id = :idval' --expression-attribute-values '{":idval":{"S":"91"}}'
You have to use value substitution on attributes. You can optionally use attribute name substitution.
Note i've assume your id attribute is of type String. Change it to "N" if its a number.
In AWS DynamoDB you must specify a `partition key, which make it just work like GetItem ... because partition key is unique so it is supposed to return only one item, so if I know the ID of that item, it make no sense anymore to query! because query meant to be for constrains ...
so can someone give me example where querying one partition key can return multiple items ?
# Create single-attribute primary key table
aws dynamodb create-table --table-name testdb6 --attribute-definitions '[{"AttributeName": "Id", "AttributeType": "S"}]' --key-schema '[{"AttributeName": "Id", "KeyType": "HASH"}]' --provisioned-throughput '{"ReadCapacityUnits": 5, "WriteCapacityUnits": 5}'
# Populate table
aws dynamodb put-item --table-name testdb6 --item '{ "Id": {"S": "1"}, "LastName": {"S": "Lopez"}, "FirstName": {"S": "Maria"}}'
aws dynamodb put-item --table-name testdb6 --item '{ "Id": {"S": "2"}, "LastName": {"S": "Fernandez"}, "FirstName": {"S": "Augusto"}}'
# Query table using only partition attribute
aws dynamodb query --table-name testdb6 --select ALL_ATTRIBUTES --key-conditions '{"Id": {"AttributeValueList": [{"S": "1"}], "ComparisonOperator": "EQ"}}'
You also can only use the EQ Operator for partition key, so using for example BETWEEN or OR or IN is not allowed on partition key
alternative to query there is scan but
scan is expensive (slow)
you can't sort on scan
Update
so I realized I can use sort key, and then in this case partition key can be my table name, so I need to change my vocabularies
Table -> Database
Partition Key -> Table / Collection
Sort Key -> Primary Key / ObjectId
Example : table my-api with partition key -> className and sort key -> id
my-api
className | id | username | title
_User | 0 | "bingo" |
_User | 1 | "mimi" |
_Song | 0 | | "You with me"
it is weird design
If the table has both partition key and sort key and query by partition key alone will give you multiple items.
Partition key - must for using Query API
Sort key - optional while using Query API
Get API:-
For a composite primary key, you must provide values for both the
partition key and the sort key.
So, Get API will always return only one item. Also, there is no filter expression to filter by non-key attributes.
Querying is for finding Items based on Primary or Index Key attributes. Primary Key can have one Partition Key (hash key) and one optional Range Key. If Range key present, then you can have multiple records with same Partition Key. So to perform GetItem operation on this kind of composite primary key you need to specify Range Key with Hash key.
Also, You can specify multiple Global Secondary Indexes (GSI) and Local Secondary Indexes (LSI), through which you can query non-key attributes.
So, Query operation provides you means to find items based on either Primary key, LSI or GSI's hash key and Range key's attributes.
Now, the example you used, is certainly not a good approach to design your schema.
Table != Database
Table == Table and DynamoDB houses all the tables.
User, Song etc needs to be stored in different tables. Then you can specify id, username to further uniquely identify items of your table. Each item can be thought of as record in RDBMS. Read more about choosing primary key and indexes and all information from AWS DynamoDB Developer Guide
Partition key would always return unique result if there is no sort key defined.
Configuring sort key to your table means that the combination between partition key and sort key must be unique.
For best practise defining the structure of your table please refer to this