I'm still learning DynamoDB and I would like to get a single item from a table based on two unique attributes (tconst and primaryTitle) in the table. Both of those attributes have unique values for each row. The primaryKey is tconst and the sortKey is primaryTitle
I thought that I could do something like below:
aws dynamodb query \
--endpoint-url http://localhost:8000 \
--table-name title \
--key-condition-expression "tconst = :tconst" and "primaryTitle = :primaryTitle" \
--expression-attribute-values '{
":tconst":{"S":"xxxx"},
":primaryTitle":{"S":"something"}
}'
"Item": {
"tconst": {
"S": "xxxx"
},
"titleType": {
"S": "xxxx"
},
"primaryTitle": {
"S": "something"
},
"originalTitle": {
"S": "Travel Daze"
},
"isAdult": {
"S": "0"
},
"startYear": {
"S": "2019"
},
"endYear": {
"S": "\\N"
},
"runtimeMinutes": {
"S": "\\N"
},
"genres": {
"S": "\\N"
}
},
"Item": {
"tconst": {
"S": "yyyy"
},
"titleType": {
"S": "yyyy"
},
"primaryTitle": {
"S": "Travel Daze"
},
"originalTitle": {
"S": "Travel Daze"
},
"isAdult": {
"S": "0"
},
"startYear": {
"S": "2019"
},
"endYear": {
"S": "\\N"
},
"runtimeMinutes": {
"S": "\\N"
},
"genres": {
"S": "\\N"
}
}
If you know the primary key (i.e. partition and sort key in your case) for your item and only want to fetch one anyway, the GetItem API call does exactly that. Query allows you to filter inside of an item collection (all items that share the same partition key) - it has additional functionality, which you may not need here.
The GetItem operation returns a set of attributes for the item with the given primary key. If there is no matching item, GetItem does not return any data and there will be no Item element in the response.
— docs
If you want to keep using the Query - you can limit the number of results using the aptly names Limit-Parameter.
Related
The bounty expires in 14 hours. Answers to this question are eligible for a +50 reputation bounty.
Wesley van S is looking for an answer from a reputable source:
Should be an easy question for someone experienced in AWS Api Gateway / DynamoDB
Quite new to AWS / DynamoDB. I'm trying to make a UpdateItem method work on my API. I have a Resource with the path x/users/update/{wallet} with wallet being the key.
In here I set a UpdateItem method just like I have all my other Query / Scan / PutItem methods. this has the following mapping template:
{
"TableName": "Users",
"KeyConditionExpression": "wallet = :v1",
"ExpressionAttributeValues": {
":v1": {
"S": "$input.params('wallet')"
}
":v2": {
"S": $input.json('$.username')
}
}
"UpdateExpression": "set username = :v2",
"ReturnValues": "UPDATED_NEW"
}
This returns a 200 with "__type": "com.amazon.coral.service#SerializationException"
Pretty sure this is related to my Mapping template but can't figure out what. Looked at the https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_UpdateItem.html and think I used the Sample Request in there as a template to test it like this
{
"TableName": "Users",
"Key": {
"wallet": {
"S": "$input.params('wallet')"
},
},
"UpdateExpression": "SET username = :val1",
"ExpressionAttributeValues": {
":val1": {
"S": "$input.path('$.username')"
},
},
"ReturnValues": "ALL_NEW"
}
payload is just as simple as
{"username":"test"}
but that gives the same result.
Any ideas on what i'm doing wrong here?
Cheers!
EDIT:
PutItem looks like this:
{
"TableName": "Users",
"Item": {
"userId": {
"S": "$context.requestId"
},
"date": {
"S": "$input.path('$.date')"
},
"wallet": {
"S": "$input.path('$.wallet')"
}
}
}
It looks like you're not accessing the record to update with the key of your table. Unlike SQL, you cannot access a record to update with any field in DynamoDB. You have to use your key depending on how you defined it (PK or PK + SK).
Given that you have both PK and SK defined, you need them both in your key input. Together they form a composite primary key. The below input should work for you.
{
"TableName": "Users",
"Key": {
"wallet": {
"S": "$input.params('wallet')"
},
"date": {
"S": "$input.params('date')"
},
},
"UpdateExpression": "SET username = :val1",
"ExpressionAttributeValues": {
":val1": {
"S": "$input.path('$.username')"
},
},
"ReturnValues": "ALL_NEW"
}
DynamoDB UpdateItem takes a Key not a KeyConditionExpression
This one below is correct:
{
"TableName": "Users",
"Key": {
"wallet": {
"S": "$input.params('wallet')"
},
},
"UpdateExpression": "SET username = :val1",
"ExpressionAttributeValues": {
":val1": {"S": "$input.params('$.username')"},
},
"ReturnValues": "ALL_NEW"
}
You also state your payload is simply
{"username":"test"}
You must also specify wallet as this is the key
I have 2 tables, with 1 matching data in which I want to utilize that as a matching field and retrieve some data.
First table is this:
{
"mainFieldName": {
"S": "someString"
},
"fieldA": {
"L": [
{
"M": {
"AccountId": {
"S": "12345"
},
"PrincipalId": {
"S": "randomIdString"
},
"PrincipalType": {
"S": "GROUP"
}
}
},
{
"M": {
"AccountId": {
"S": "12345"
},
"PrincipalId": {
"S": "secondRandomString"
},
"PrincipalType": {
"S": "GROUP"
}
}
}
]
},
"fieldC": {
"L": [
{
"M": {
"name": {
"S": "xxx"
},
"final_json": {
"S": "some json data"
}
}
}
]
}
}
Second table:
{
"userId": {
"S": "randomString"
},
"group": {
"L": [
{
"M": {
"GroupId": {
"S": "randomGroupId"
}
}
}
]
}
}
I want to find the matched field for first table's fieldA.PrincipalId and second table's group.GroupId, if match, returning data is first table's fieldC.final_json
My params i tried is this, it's executed successfully but no results returned. I have confirmed there should be some matched inputs.
response = table1.scan(
TableName=TABLE_1,
FilterExpression="#gid.#pid = :id",
ExpressionAttributeValues={
':id': {'S': groupId}
},
ExpressionAttributeNames={
'#gid': 'groupId',
'#pid': 'PrincipalId'
}
)
It always return empty results
I managed to find a resolution to this. To simplify, I changed to a flatter table structure by pre-processing the Json to appending to a list.
My first table becomes:
{
"id": {
"S": "randomString"
},
"fieldA": {
"S": "randomString"
},
"table_1_groupId": {
"L": [
{
"S": "randomGroupIdString"
}
]
},
"fieldB": {
"S": "asdfsafd"
},
"fieldC": {
"L": [
{
"M": {
"name": {
"S": "randomString"
},
"jsonData": {
"S": "randomJsonData"
},
"type": {
"S": "type_a"
}
}
}
]
}
}
Second table stays the same.
With that i am able to use DynamoDB query which is more efficient as well, with FilterExpressions
My query is:
response = table1.query(
TableName=TABLE_1,
KeyConditionExpression="id = :id",
FilterExpression="contains(groupId, :gid)",
ExpressionAttributeValues={
':id': defaultId,
':gid': groupId
},
)
My output returns list of all data (since I haven't added any filter to the output) once they have the field 'GroupId' in Table2 match with table_1_groupId in table 1
I have this dynamodb:Query in my step function:
{
"Type": "Task",
"Resource": "arn:aws:states:::aws-sdk:dynamodb:query",
"Next": "If nothing returned by query Or Study not yet Zipped",
"Parameters": {
"TableName": "TEST-StudyProcessingTable",
"ScanIndexForward": false,
"Limit": 1,
"KeyConditionExpression": "OrderID = :OrderID",
"FilterExpression": "StudyID = :StudyID",
"ExpressionAttributeValues": {
":OrderID": {
"S.$": "$.body.order_id"
},
":StudyID": {
"S.$": "$.body.study_id"
}
}
},
"ResultPath": "$.processed_files"
}
The results comes in as an array called Items which is nested under my ResultPath
processed_files.Items:
{
"body": {
"order_id": "1001",
"study_id": "1"
},
"processed_files": {
"Count": 1,
"Items": [
{
"Status": {
"S": "unzipped"
},
"StudyID": {
"S": "1"
},
"ZipFileS3Key": {
"S": "path/to/the/file"
},
"UploadSet": {
"S": "4"
},
"OrderID": {
"S": "1001"
},
"UploadSet#StudyID": {
"S": "4#1"
}
}
],
"LastEvaluatedKey": {
"OrderID": {
"S": "1001"
},
"UploadSet#StudyID": {
"S": "4#1"
}
},
"ScannedCount": 1
}
}
My question is how do i access the items inside this array from a choice state in a step function?
I need to query then decide something based on the results by checking the item in a condition in a choice state.
The problem is that since this is an array I can't access it using regular JsonPath (like with Items.item), and in my next step the choice condition does NOT accept an index like processed_files.Items['0'].Status
Ok so the answer was so simple all you need to do is use a number instead of string for the array index like this.
processed_files.Items[0].Status
I was originally mislead by an error I received which said that it expected a ' or '[' after the first '['. I mistakenly thought this meant it only accepts strings.
I was wrong, it works like any other array.
I hope this helps somebody one day.
In one of my AWS DynamoDB tables I have several items that require an update within in a pipeline.
The following json shows an item in the AWS DynamoDB table:
{
"DeploymentConfigs": {
"L": [
{
"M": {
"Config": {
"M": {
"Name": {
"S": "batch-komo"
},
"Replicas": {
"N": "3"
}
}
}
}
},
{
"M": {
"Config": {
"M": {
"Name": {
"S": "online-komo"
},
"Replicas": {
"N": "3"
}
}
}
}
}
]
},
"environment": {
"S": "komo-claimcenter"
}
}
How can update an object in DeploymentConfigs?
Primary partition key is environment.
E.g. the object
{
"M": {
"Config": {
"M": {
"Name": {
"S": "batch-komo"
},
"Replicas": {
"N": "3"
}
}
}
}
}
shall be updated to
{
"M": {
"Config": {
"M": {
"Name": {
"S": "batch-komo"
},
"Replicas": {
"N": "5"
}
}
}
}
}
I do not know to achieve in the AWS CLI.
To update a nested part inside an item, you use an UpdateExpression with an attribute path. For example, SET DeploymentConfigs[0].Config.Replicas = :val.
The problem, however, is that your top-level attribute DeploymentConfigs is a list, so to modify one of its items, you need to know in index (in this example 0). If you don't know the index you have a problem - there is no way to "modify the list item which has name='batch-komo'" or something like that... If you need to do something like this, you have to read the entire top-level attribute, modify it in the client, and write it (or just the small change, now that you know the index) back.
tl;dr looking to sort API output from AWS API gateway
I have a dynamoDB table 'NP' with key 'id' and GSI 'atype-index'. There is no sort key.
Also, I have API gateway mapping template
{
"TableName": "NP",
"IndexName": "atype-index",
"KeyConditionExpression": "atype = :v1",
"ExpressionAttributeValues": {
":v1": {
"S": "$input.params('atype')"
}
}
}
The mapping template will extract all values in the table with GSI key matching the specified string. However, the values are extracted in no particular order, for example, the API could be:
{
"Count": 3,
"Items": [
{
"apiUrl": {
"S": “ee”
},
"webTitle": {
"S": “dd”
},
"atype": {
"S": “type”
},
"id": {
"S": "1a"
}
},
{
"apiUrl": {
"S": “dd”
},
"webTitle": {
"S": “cc”
},
"atype": {
"S": "atype"
},
"id": {
"S": “3a”
}
},
{
"apiUrl": {
"S": “cc”
},
"webTitle": {
"S": “bb”
},
"atype": {
"S": "atype"
},
"id": {
"S": “2a”
}
}
],
"ScannedCount": 3
}
How do I sort the above output by the key "apiURL" for example?
What you described is exactly what a sort key would accomplish, assuming you're using the Query API (rather than Scan).
If you add a sort key, the results will be sorted based on the sort key when querying a given partition key.
Edit: Just to clarify, there is no way to do the sort at the API GW layer in a mapping template.
You could put a Lambda function in between API GW and DDB if the overhead is worth it for your use case. Otherwise, probably better to just sort at client side.