How to update a nested item in AWS DynamoDB with CLI - amazon-web-services

In one of my AWS DynamoDB tables I have several items that require an update within in a pipeline.
The following json shows an item in the AWS DynamoDB table:
{
"DeploymentConfigs": {
"L": [
{
"M": {
"Config": {
"M": {
"Name": {
"S": "batch-komo"
},
"Replicas": {
"N": "3"
}
}
}
}
},
{
"M": {
"Config": {
"M": {
"Name": {
"S": "online-komo"
},
"Replicas": {
"N": "3"
}
}
}
}
}
]
},
"environment": {
"S": "komo-claimcenter"
}
}
How can update an object in DeploymentConfigs?
Primary partition key is environment.
E.g. the object
{
"M": {
"Config": {
"M": {
"Name": {
"S": "batch-komo"
},
"Replicas": {
"N": "3"
}
}
}
}
}
shall be updated to
{
"M": {
"Config": {
"M": {
"Name": {
"S": "batch-komo"
},
"Replicas": {
"N": "5"
}
}
}
}
}
I do not know to achieve in the AWS CLI.

To update a nested part inside an item, you use an UpdateExpression with an attribute path. For example, SET DeploymentConfigs[0].Config.Replicas = :val.
The problem, however, is that your top-level attribute DeploymentConfigs is a list, so to modify one of its items, you need to know in index (in this example 0). If you don't know the index you have a problem - there is no way to "modify the list item which has name='batch-komo'" or something like that... If you need to do something like this, you have to read the entire top-level attribute, modify it in the client, and write it (or just the small change, now that you know the index) back.

Related

Match non-partitionKey fields (in nested json) in both tables and retrieve data in DynamoDB table

I have 2 tables, with 1 matching data in which I want to utilize that as a matching field and retrieve some data.
First table is this:
{
"mainFieldName": {
"S": "someString"
},
"fieldA": {
"L": [
{
"M": {
"AccountId": {
"S": "12345"
},
"PrincipalId": {
"S": "randomIdString"
},
"PrincipalType": {
"S": "GROUP"
}
}
},
{
"M": {
"AccountId": {
"S": "12345"
},
"PrincipalId": {
"S": "secondRandomString"
},
"PrincipalType": {
"S": "GROUP"
}
}
}
]
},
"fieldC": {
"L": [
{
"M": {
"name": {
"S": "xxx"
},
"final_json": {
"S": "some json data"
}
}
}
]
}
}
Second table:
{
"userId": {
"S": "randomString"
},
"group": {
"L": [
{
"M": {
"GroupId": {
"S": "randomGroupId"
}
}
}
]
}
}
I want to find the matched field for first table's fieldA.PrincipalId and second table's group.GroupId, if match, returning data is first table's fieldC.final_json
My params i tried is this, it's executed successfully but no results returned. I have confirmed there should be some matched inputs.
response = table1.scan(
TableName=TABLE_1,
FilterExpression="#gid.#pid = :id",
ExpressionAttributeValues={
':id': {'S': groupId}
},
ExpressionAttributeNames={
'#gid': 'groupId',
'#pid': 'PrincipalId'
}
)
It always return empty results
I managed to find a resolution to this. To simplify, I changed to a flatter table structure by pre-processing the Json to appending to a list.
My first table becomes:
{
"id": {
"S": "randomString"
},
"fieldA": {
"S": "randomString"
},
"table_1_groupId": {
"L": [
{
"S": "randomGroupIdString"
}
]
},
"fieldB": {
"S": "asdfsafd"
},
"fieldC": {
"L": [
{
"M": {
"name": {
"S": "randomString"
},
"jsonData": {
"S": "randomJsonData"
},
"type": {
"S": "type_a"
}
}
}
]
}
}
Second table stays the same.
With that i am able to use DynamoDB query which is more efficient as well, with FilterExpressions
My query is:
response = table1.query(
TableName=TABLE_1,
KeyConditionExpression="id = :id",
FilterExpression="contains(groupId, :gid)",
ExpressionAttributeValues={
':id': defaultId,
':gid': groupId
},
)
My output returns list of all data (since I haven't added any filter to the output) once they have the field 'GroupId' in Table2 match with table_1_groupId in table 1

How to add new attributes to map in DynamoDB?

My database structure,
{
"email":"example#mail.com",
"products": {
"product1":{
"price":"$10",
"details":"detail"
},
"product2":{
"price":"$20",
"details":"detail"
}
}
}
I want to add new attributes to "products" map and expected output as follow,
{
"email":"example#mail.com",
"products": {
"product1":{
"price":"$10",
"details":"detail"
},
"product2":{
"price":"$20",
"details":"detail"
},
"product3":{
"price":"$10",
"details":"detail"
}
}
}
I am using API Gateway and UpdateItem action. Here is my mapping template,
{
"TableName": "tableName",
"Key": {
"email": {
"S": "$input.path('$.email')"
}
},
"UpdateExpression": "SET #pr = :vals",
"ExpressionAttributeNames": {
"#pr": "products"
},
"ExpressionAttributeValues": {
":vals": {
"M": {
"$input.path('$.productId')": {
"M": {
"price": {
"N": "$input.path('$.price')"
},
"details": {
"S": "$input.path('$.details')"
}
}
}
}
}
},
"ReturnValues": "NONE"
}
Using above template will replace all my attributes. Actual output,
{
"email":"example#mail.com",
"products": {
"product3":{
"price":"$10",
"details":"detail"
}
}
}
How I can add new attributes to map instead of replace it?
Thanks.
In your request you are SETting the entire Products map, but you only want to add a nested map.
{
"TableName": "tableName",
"Key": {
"email": {
"S": "$input.path('$.email')"
}
},
"UpdateExpression": "SET #pr.product3 = :vals",
"ExpressionAttributeNames": {
"#pr": "products"
},
"ExpressionAttributeValues": {
":vals": {
"M": {
"$input.path('$.productId')": {
"M": {
"price": {
"N": "$input.path('$.price')"
},
"details": {
"S": "$input.path('$.details')"
}
}
}
}
}
},
"ReturnValues": "NONE"
}

Access an Array Item by index in AWS Dynamodb Query Results "Items" in Step Function

I have this dynamodb:Query in my step function:
{
"Type": "Task",
"Resource": "arn:aws:states:::aws-sdk:dynamodb:query",
"Next": "If nothing returned by query Or Study not yet Zipped",
"Parameters": {
"TableName": "TEST-StudyProcessingTable",
"ScanIndexForward": false,
"Limit": 1,
"KeyConditionExpression": "OrderID = :OrderID",
"FilterExpression": "StudyID = :StudyID",
"ExpressionAttributeValues": {
":OrderID": {
"S.$": "$.body.order_id"
},
":StudyID": {
"S.$": "$.body.study_id"
}
}
},
"ResultPath": "$.processed_files"
}
The results comes in as an array called Items which is nested under my ResultPath
processed_files.Items:
{
"body": {
"order_id": "1001",
"study_id": "1"
},
"processed_files": {
"Count": 1,
"Items": [
{
"Status": {
"S": "unzipped"
},
"StudyID": {
"S": "1"
},
"ZipFileS3Key": {
"S": "path/to/the/file"
},
"UploadSet": {
"S": "4"
},
"OrderID": {
"S": "1001"
},
"UploadSet#StudyID": {
"S": "4#1"
}
}
],
"LastEvaluatedKey": {
"OrderID": {
"S": "1001"
},
"UploadSet#StudyID": {
"S": "4#1"
}
},
"ScannedCount": 1
}
}
My question is how do i access the items inside this array from a choice state in a step function?
I need to query then decide something based on the results by checking the item in a condition in a choice state.
The problem is that since this is an array I can't access it using regular JsonPath (like with Items.item), and in my next step the choice condition does NOT accept an index like processed_files.Items['0'].Status
Ok so the answer was so simple all you need to do is use a number instead of string for the array index like this.
processed_files.Items[0].Status
I was originally mislead by an error I received which said that it expected a ' or '[' after the first '['. I mistakenly thought this meant it only accepts strings.
I was wrong, it works like any other array.
I hope this helps somebody one day.

Unable to map a list of numbers in API-Gateway

How do I create a mapping for a list of numbers object in API gateway? I am trying to post a list of integers using POST request. I tried working with NS attribute but got the Error.
Error:
{
"__type": "com.amazon.coral.service#SerializationException"
}
However, it works well when I have N attribute and post a single integer value.
Is there any way to resolve this issue?
I believe you are trying to map your request payload to DynamoDB JSON String. You can apply a velocity template like this one,
{
"TableName":"ABC",
"Item": {
"id": {
"S": "$context.requestId"
},
"name": {
"S": "$input.path('$.name')"
},
"price": {
"L": [
#set($prices=$input.path('$.price'))
#foreach($p in $prices)
{
"N": "$p"
}#if ($velocityCount < $prices.size()), #end
#end
]
}
}
}
Method Request Body:
{
"name":"Test",
"price": [1, 2, 3]
}
Endpoint Request Body:
{
"TableName": "ABC",
"Item": {
"id": {
"S": "test-invoke-request"
},
"name": {
"S": "Test"
},
"price": {
"L": [
{
"N": "1"
},
{
"N": "2"
},
{
"N": "3"
}
]
}
}
}

How do I configure an UpdateExpression with nested data

I have an AWS API Gateway that stores data in a DynamoDB instance. My table structure looks like this:
{
"TableName": "stuff",
"Item": {
"stuffId": {
"S": "02b4e004-1132-4b87-a855-20e7d1bd1840"
},
"clients": {
"M": {
"company_inc": {
"M": {
"prod": {
"S": "null"
},
"qa": {
"S": "null"
},
"stage": {
"S": "null"
}
}
}
}
}
}
}
I'm trying to figure out how to configure my Body Mapping Template so that given an HTTP PATCH request I can update company_inc.prod. For example; given this query string:
?stuffId=02b4e004-1132-4b87-a855-20e7d1bd1840&client=company_inc&location=prod&locationIsSet=true
I would update the record to look like this:
{
"TableName": "stuff",
"Item": {
"stuffId": {
"S": "02b4e004-1132-4b87-a855-20e7d1bd1840"
},
"clients": {
"M": {
"company_inc": {
"M": {
"prod": {
"S": "true"
},
"qa": {
"S": "null"
},
"stage": {
"S": "null"
}
}
}
}
}
}
}
What should an "UpdateExpression" look like to achieve that?
I think I found the answer. You have to use attribute names like #client and #location as placeholders in the path for company_inc.prod.
{
"TableName": "stuff",
"Key": {
"alterId": {
"S": "$input.params('stuffId')"
}
},
"UpdateExpression": "set clients.#attrClientName.#attrLocation = :locationIsSet",
"ExpressionAttributeNames" : {
"#attrClientName" : "$input.params('client')",
"#attrLocation" : "$input.params('location')"
},
"ExpressionAttributeValues": {
":locationIsSet": {"S": "$input.params('location')"}
}
}