How to get data from aws Dynamodb with using partition key only? - amazon-web-services

I am using aws-sdk-go library for DynamoDb connectivity in Golang.
My DynamoDb table have a Partition key DeviceId (String) and a Sort Key Time (Number). How can I write GetItemInput to get all data with a specific DeviceId?
params := &dynamodb.GetItemInput{
Key: map[string]*dynamodb.AttributeValue {
"DeviceId": {
S: aws.String("item_1"),
},
},
ExpressionAttributeNames: map[string]*string{
"DeviceId": "DeviceId",
},
TableName: aws.String("DbName"),
}
list, err := svc.GetItem(params)

You have to use Query or Scan operation, this is a simple example but you can read more on Amazon documentation here
In particular, Query operation
A Query operation finds items in a table or a secondary index using only primary key attribute values
var queryInput = &dynamodb.QueryInput{
TableName: aws.String(dynamoRestDataTableName),
KeyConditions: map[string]*dynamodb.Condition{
"DeviceId": {
ComparisonOperator: aws.String("EQ"),
AttributeValueList: []*dynamodb.AttributeValue{
{
S: aws.String("aDeviceId"),
},
},
},
},
}
var resp, err = dynamoSvc.Query(queryInput)
if err != nil {
return nil, err
}

Query operation can be used in that case
Following is one generic example for the same
compositeKey := entity.GetPrimaryKey(inputVar)
expressionAttributeValues := map[string]*dynamodb.AttributeValue{
":v1": {
S: aws.String(compositeKey.PartitionKey.Data.(string)),
},
}
queryInput := dynamodb.QueryInput{
TableName: &d.TableName,
KeyConditionExpression: aws.String("id = :v1"),
ExpressionAttributeValues: expressionAttributeValues,
}
queryOutput, err := d.DdbSession.Query(&queryInput)
if err != nil {
log.Error("error in fetching records ", err)
return nil, err
}
// unmarshal the query output - items to interface
err = dynamodbattribute.UnmarshalListOfMaps(queryOutput.Items, &yourInterface)

You can use getItem in JavaScript SDK v2 like this
const params = {
TableName: 'tableName',
Key: {
id: { S: id },
},
};
const result = await dynamoDb.getItem(params).promise();
if (result.Item === undefined) {
throw new Error('not found');
}
const item = AWS.DynamoDB.Converter.unmarshall(result.Item)
If JavaScript SDK can do this, I assume golang SDK can also. If not, that means AWS doesn't take all languages equally ?

Related

DynamoDB return the modified document (Old or new) when using TransactWrtieItems

Is there a way of making transactWriteItem return the document it updated?
const transactionParams = {
ReturnConsumedCapacity: "INDEXES",
TransactItems: [
{
Delete: {
TableName: reactionTableName,
Key: {
"SOME_PK_",
"SOME_SK_",
},
ReturnValues: 'ALL_OLD',
},
},
{
Update: {
TableName: reviewTableName,
Key: { PK: "SOME_PK", SK: "SOME_SK" },
ReturnValues: 'ALL_OLD',
},
},
],
};
try {
const result = await docClient.transactWrite(transactionParams).promise();
} catch (error) {
context.done(error, null);
}
For example in the above code get the documents that were touched (before or after update)?
No, TransactWriteItems API does not provide the ability to return values of a modified item, however, you could obtain those values using DynamoDB Streams, otherwise you would need to default to the singleton APIs UpdateItem/DeleteItem which are not ACID compliant together.

Querying with FilterExpression CONTAINS method

I want to query a specific row from a table based on a track's genre and release year.
Task: Query rows with release_year=2018 AND genre="pop"
Tracks Table:
trackId----------------release_year----------------genres----------------count
trackId: "7sT7kZEYd1MrmzLLIRVZas", release_year: 2018, genres: ["pop","rap", "hip-hop"], count: 7
Below code is how I am attempting to make this query as shown in FilterExpression. Currently I'm checking if the Table's attribute: genres (Array), contains :genre, which is a String sent from the client in the request. How would one set up a GSI in a way to make this query?
let queryParams = {
TableName: tableName,
IndexName: req.query.index,
KeyConditionExpression: 'release_year = :release_year and genres = :genre',
ExpressionAttributeValues: { ':release_year': parseInt(req.query.year), ':genre': req.query.genre},
FilterExpression : "contains (genres, :genre)",
ScanIndexForward: false,
Limit: 50
}
dynamodb.query(queryParams, (err, data) => {
if (err) {
res.statusCode = 500;
res.json({error: 'Could not load items: ' + err});
} else {
res.json(data.Items);
}
});
GSI should be created with release_year as the PK. Generes is part of the FilterExpression only.
let queryParams = {
TableName: tableName,
IndexName: req.query.index,
KeyConditionExpression: 'release_year = :release_year',
ExpressionAttributeValues: { ':release_year': parseInt(req.query.year), ':genre': req.query.genre},
FilterExpression : "contains (genres, :genre)",
ScanIndexForward: false,
Limit: 50
}
dynamodb.query(queryParams, (err, data) => {
if (err) {
res.statusCode = 500;
res.json({error: 'Could not load items: ' + err});
} else {
res.json(data.Items);
}
});

How can I update a nested field in DynamoDB?

I defined a DynamoDB table and it has a nested field site. I'd like to update the field site.enable in below code. But when I run the update command I got this error:
ValidationException: The document path provided in the update expression is invalid for update`
What should I do in order to fix the issue?
{
TableName: 'MyTable',
Key: {
id: '4b7020d2-2d19-4aeb-7f27e49d5bec',
type: '80422149-c97d-4a1a-7bf20ef57056',
},
UpdateExpression: 'set #site.#siteenable= :siteenable',
ExpressionAttributeValues: {
':siteenable': true,
},
ExpressionAttributeNames: {
'#siteenable': 'enable',
'#site': 'site',
}
}
You don't mention a programming language, so I'm going to assume what I'm used to: Python.
In Python there are two ways you can do this:
The lower level client API, which requires you to format the data the way DynamoDB would
def enable_site_with_client():
ddb = boto3.client("dynamodb")
ddb.update_item(
TableName=TABLE_NAME,
Key={
"PK": {"S": "SITE_ENTRY"}
},
UpdateExpression="SET #site.#enabled = :update_value",
ExpressionAttributeNames={
"#site": "site",
"#enabled": "enabled"
},
ExpressionAttributeValues={
":update_value": {"BOOL": True}
}
)
The higher level resource API, which allows you to use the language native data structures
def enable_site_with_resource():
ddb = boto3.resource("dynamodb")
ddb.Table(TABLE_NAME).update_item(
Key={
"PK": "SITE_ENTRY"
},
UpdateExpression="SET #site.#enabled = :update_value",
ExpressionAttributeNames={
"#site": "site",
"#enabled": "enabled"
},
ExpressionAttributeValues={
":update_value": True
}
)
I have tested both of these and they work.
Given code works fine if the map site exists already, seeing the error message, it looks like the path site doesn't exist.
We could create empty map during the create of the document, then update it easily OR
We could create the map during the update "set #site = :siteValue"
Here is slightly modified query which creates the map.
const dynamodb = new AWS.DynamoDB();
let docClient = new AWS.DynamoDB.DocumentClient();
docClient.update(
{
TableName: "MyTable",
Key: {
id: "4b7020d2-2d19-4aeb-7f27e49d5bec",
type: "80422149-c97d-4a1a-7bf20ef57056",
},
UpdateExpression: "set #site = :siteValue",
ExpressionAttributeValues: {
":siteValue": { enable: true },
},
ExpressionAttributeNames: {
"#site": "site",
},
},
function (error, result) {
console.log("error", error, "result", result);
}
);
Here is an example for Java SDK V2 that will update root->nested->someValue
Map<String, AttributeValue> attributeValues = new HashMap<>();
attributeValues.put(":myValue", AttributeValue.builder().s(jsonString).build());
UpdateItemRequest updateRequest = UpdateItemRequest.builder()
.tableName("my_table_name")
.key(keyToUpdate)
.updateExpression("SET nested.someValue= :myValue")
.expressionAttributeValues(attributeValues)
.build();
client.updateItem(updateRequest);

DynamoDB returning null nextToken when there are still results in database

I have a GraphQL API (AppSync) backed by a DynamoDB table keyed a specific id with timestamp as the range key. I want to retrieve all possible history for that id so I wrote a query in my GraphQL schema that would allow me to do so. Here's the request vtl:
{
"version": "2017-02-28",
"operation": "Query",
"query": {
"expression": "id = :id",
"expressionValues": {
":id": {
"S": "$context.args.id"
}
}
},
"nextToken": $util.toJson($util.defaultIfNullOrEmpty($context.args.nextToken, null))
}
There could be thousands of items in the ddb table for an id so I wrote a Lambda function to query for all of them and return the result in a list as such (I know the code can be simplified):
exports.handler = async function (event, context, callback) {
const graphqlClient = new appsync.AWSAppSyncClient({
url: process.env.APPSYNC_ENDPOINT,
region: process.env.AWS_REGION,
auth: {
type: 'AWS_IAM',
credentials: {
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
sessionToken: process.env.AWS_SESSION_TOKEN
}
},
disableOffline: true
});
const query = gql`query GetAllItemsById(
$id: String!
$nextToken: String
) {
getAllItemsById(
id: $id
nextToken: $nextToken
) {
exampleField {
subField1
subField2
subField3
}
nextToken
}
}
`;
const initialResult = await graphqlClient.query({
query,
variables: {
id: event.id
}
});
var finalResult = initialResult.data.getAllItemsById.exampleField;
var nextToken = initialResult.data.getAllItemsById.nextToken;
while (nextToken !== null) {
const result = await graphqlClient.query({
query,
variables: {
id: event.id,
nextToken: nextToken
}
});
finalResult = finalResult.concat(result.data.getAllItemsById.exampleField);
nextToken = result.data.getAllItemsById.nextToken;
}
console.log("Total Results: " + finalResult.length);
return callback(null, finalResult);
};
For some reason, not all items are being returned. nextToken is null before all results are returned. I know DDB has a 1MB limit for query which is why I'm paginating using nextToken but why is it still not returning all the items in the table? Also, if there's a better way to implement this, I'm open to it.

AWS DynamoDB Attempting to ADD to a Set - Incorrect Operand

I am creating an API using Nodejs and DynamoDB as a back end. I am attempting to update an item to add to a set of "friends". When I update the user, I get the error, "Invalid UpdateExpression: Incorrect operand type for operator or function; operator: ADD, operand type: MAP". My understanding is that when adding to a set that does not exist, the set will be created. If it already exists, the new value should be added to the set. I do not understand why the set I attempt to ADD is being read as a map.
How users are created:
var params = {
TableName: "users",
Item:{
"id": Number(id),
"name": name,
"password": password
}
};
documentClient.put(params, function(err, data) {
if(err)
res.json(500, err);
else
res.json(200, data);
});
How friends are added:
var params = {
TableName: "users",
Key: {
"id": id
},
UpdateExpression: "ADD friends :friendId",
ExpressionAttributeValues: {
":friendId": { "NS": [friendId] }
},
ReturnValues: "UPDATED_NEW"
};
documentClient.update(params, function(err, data) {
if(err)
res.json(500, err);
else
res.json(200, data);
});
This question has an answer here
https://stackoverflow.com/a/38960676/4975772
Here's the relevant code formatted to fit your question
let AWS = require('aws-sdk');
let docClient = new AWS.DynamoDB.DocumentClient();
...
var params = {
TableName : 'users',
Key: {'id': id},
UpdateExpression : 'ADD #friends :friendId',
ExpressionAttributeNames : {
'#friends' : 'friends'
},
ExpressionAttributeValues : {
':friendId' : docClient.createSet([friendId])
},
ReturnValues: 'UPDATED_NEW'
};
docClient.update(params, callback);
If the set doesn't exist, then that code will create it for you. You can also run that code with a different set to update the set's elements. Super convenient.
Here is the working code. You don't need ADD here. Just use "set friends = :friendId" as friends attribute is not already present in the table (i.e. before the update you have only id, name and password in the table). The friend attribute is being added newly as part of the update.
var docClient = new AWS.DynamoDB.DocumentClient();
var table = "users";
var userid = 1;
var friendId = [123];
var params = {
TableName : table,
Key: {
"id" : userid
},
"UpdateExpression": "set friends = :friendId",
"ExpressionAttributeValues": {
":friendId": {"NS": friendId}
},
"ReturnValues" : "UPDATED_NEW"
};