Update specific attribute using UpdateItemEnhancedRequest DynamoDb java sdk2 - amazon-web-services

We have a DynamoDB table which has an attribute counter, which will be decremented asynchronously by multiple lambda based on an event. I am trying to update the counter using UpdateItemEnhancedRequest (using the Dynamodb Enhanced Client. - JAVA SDK 2). I am able to build the condition for updating the counter but it updates the entire item and not just the counter. Can somebody please guide on how to update a single attribute using DynamoDb Enhanced Client?
Code Sample
public void update(String counter, T item) {
AttributeValue value = AttributeValue.builder().n(counter).build();
Map<String, AttributeValue> expressionValues = new HashMap<>();
expressionValues.put(":value", value);
Expression myExpression = Expression.builder()
.expression("nqctr = :value")
.expressionValues(expressionValues)
.build();
UpdateItemEnhancedRequest<T> updateItemEnhancedRequest =
UpdateItemEnhancedRequest.builder(collectionClassName)
.item(item)
.conditionExpression(myExpression)
.build();
getTable().updateItem(updateItemEnhancedRequest);
}

When you update a specific column, you need to specify which column to update. Assume we have this table:
Now assume we want to update the archive column. You need to specify the column in your code. Here we change the archive column of the item that corresponds to the key to Closed (a single column update). Notice we specify the column name by using the HashMap object named updatedValues.
// Archives an item based on the key
public String archiveItem(String id){
DynamoDbClient ddb = getClient();
HashMap<String,AttributeValue> itemKey = new HashMap<String,AttributeValue>();
itemKey.put("id", AttributeValue.builder()
.s(id)
.build());
HashMap<String, AttributeValueUpdate> updatedValues =
new HashMap<String,AttributeValueUpdate>();
// Update the column specified by name with updatedVal
updatedValues.put("archive", AttributeValueUpdate.builder()
.value(AttributeValue.builder()
.s("Closed").build())
.action(AttributeAction.PUT)
.build());
UpdateItemRequest request = UpdateItemRequest.builder()
.tableName("Work")
.key(itemKey)
.attributeUpdates(updatedValues)
.build();
try {
ddb.updateItem(request);
return"The item was successfully archived";
NOTE: This is not the Enhanced Client.
This code is from the AWS Tutorial that show how to build a Java web app by using Spring Boot. Full tutorial here:
Creating the DynamoDB web application item tracker
TO update a single column using the Enhanced Client, call the Table method. This returns a DynamoDbTable instance. Now you can call the updateItem method.
Here is the logic to update the the archive column using the Enhanced Client. Notice you get a Work object, call its setArchive then pass the Work object. workTable.updateItem(r->r.item(work));
Java code:
// Update the archive column by using the Enhanced Client.
public String archiveItemEC(String id) {
DynamoDbClient ddb = getClient();
try {
DynamoDbEnhancedClient enhancedClient = DynamoDbEnhancedClient.builder()
.dynamoDbClient(getClient())
.build();
DynamoDbTable<Work> workTable = enhancedClient.table("Work", TableSchema.fromBean(Work.class));
//Get the Key object.
Key key = Key.builder()
.partitionValue(id)
.build();
// Get the item by using the key.
Work work = workTable.getItem(r->r.key(key));
work.setArchive("Closed");
workTable.updateItem(r->r.item(work));
return"The item was successfully archived";
} catch (DynamoDbException e) {
System.err.println(e.getMessage());
System.exit(1);
}
return "";
}
This answer shows both ways to update a single column in a DynamoDB table. The above tutorial now shows this way.

In your original solution, you're misinterpreting the meaning of the conditionExpression attribute. This is used to validate conditions that must be true on the item that matches the key in order to perform the update, not the expression to perform the update itself.
There is a way to perform this operation with the enhanced client without needing to fetch the object before making an update. The UpdateItemEnhancedRequest class has an ignoreNulls attribute that will exclude all null attributes from the update. This is false by default, which is what causes a full overwrite of the object.
Let's assume this is the structure of your item (without all the enhanced client annotations and boilerplate, you can add those):
class T {
public String partitionKey;
public Int counter;
public String someOtherAttribute
public T(String partitionKey) {
this.partitionKey = partitionKey;
this.counter = null;
this.someOtherAttribute = null
}
}
You can issue an update of just the counter, and only if the item exists, like this:
public void update(Int counter, String partitionKey) {
T updateItem = new T(partitionKey)
updateItem.counter = counter
Expression itemExistsExpression = Expression.builder()
.expression("attribute_exists(partitionKey)")
.build();
UpdateItemEnhancedRequest<T> updateItemEnhancedRequest =
UpdateItemEnhancedRequest.builder(collectionClassName)
.item(item)
.conditionExpression(itemExistsExpression)
.ignoreNulls(true)
.build();
getTable().updateItem(updateItemEnhancedRequest);
}

Related

DynamoDB Java AWS SDK - UpdateItem simplification possible?

I am new to using AWS and am writing something that will add an attribute to the provided item given some keyAttributes. I got it working, but the code I ended up writing just looks unintuitive and silly to me. (I just based it off of documentation I've been finding online.)
I was able to do a successful basic update of an item in a dynamoDB table with this code:
final AttributeValue fulfilled = new AttributeValue().withBOOL(true);
final UpdateItemRequest updateItemRequest = new UpdateItemRequest()
.withTableName(tableName)
.withKey(keyAttributes)
.withUpdateExpression("SET fulfilled = :fulfilled")
.withExpressionAttributeValues(ImmutableMap.of(":fulfilled", fulfilled));
final UpdateItemResult result = dynamoClient.updateItem(updateItemRequest);
Is there anything I can do to cut it down to just the barebones thing I'm trying to do here: just adding a new attribute "fulfilled" to an item, set to true?
An UpdateItemRequest must always contain the table name and the key attributes. (Otherwise how would DynamoDB know which item to update?)
You can simplify it, though, by getting rid of the ExpressionAttributeValues, like this:
final UpdateItemRequest updateItemRequest = new UpdateItemRequest()
.withTableName(tableName)
.withKey(keyAttributes)
.withUpdateExpression("SET fulfilled = TRUE");
final UpdateItemResult result = dynamoClient.updateItem(updateItemRequest);

Using Spanner within Apache Beam Dataflow

I am trying to add a Spanner connection within an Apache Beam ParDo(DoFn). I need to lookup some rows as part of the ParDo. The dataflow creates a number of workers (usually 4 max) and I use the startBundle and finishBundle methods to open and close the spanner connections for the workers lifetime. Then within the processElement method I perform the lookup for each item passing the DatabaseClient and using a singleUseReadOnlyTransaction.
I should add this is running as a dataflow under GCP
Some code to illustrate this.
private static CustomDoFn<String, TransactionImport> processRow = new CustomDoFn<String, TransactionImport>(){
private static final long serialVersionUID = 1L;
private Spanner spanner = null;
private DatabaseClient dbClient = null;
#StartBundle
public void startBundle(StartBundleContext c){
TransactionFileOptions options = c.getPipelineOptions().as(TransactionFileOptions.class);
com.google.cloud.spanner.SpannerOptions spannerOptions = com.google.cloud.spanner.SpannerOptions.newBuilder().build();
spanner = spannerOptions.getService();
String spannerProjectID = options.getSpannerProjectId();
String spannerInstanceID = options.getSpannerInstanceId();
String spannerDatabaseID = options.getSpannerDatabaseId();
DatabaseId db = DatabaseId.of(spannerProjectID, spannerInstanceID, spannerDatabaseID);
dbClient = spanner.getDatabaseClient(db);
}
#FinishBundle
public void finishBundle(FinishBundleContext c){
spanner.close();
}
#ProcessElement
public void processElement(DoFn<String, TransactionImport>.ProcessContext c) throws Exception {
TransactionImport import = new TransactionImport();
Statement statement = Statement.newBuilder("SELECT * FROM Table1 WHERE Name= #Name")
.bind("Name").to( text)
.build();
ResultSet resultSet = dbClient.singleUseReadOnlyTransaction().executeQuery(statement);
// set some value on import dependant on retrieved value
c.output(import);
}
This always results in the dataflow not completing and when I check the log I see:
Processing stuck in step Process Rows for at least 05m00s without outputting or completing in state process
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924)
at com.google.common.util.concurrent.Uninterruptibles.takeUninterruptibly(Uninterruptibles.java:233)
at com.google.cloud.spanner.SessionPool$Waiter.take(SessionPool.java:411)
at com.google.cloud.spanner.SessionPool$Waiter.access$3300(SessionPool.java:399)
at com.google.cloud.spanner.SessionPool.getReadSession(SessionPool.java:754)
at com.google.cloud.spanner.DatabaseClientImpl.singleUseReadOnlyTransaction(DatabaseClientImpl.java:52)
at com.mycompany.pt.SpannerDataAccess.getBinDetails(SpannerDataAccess.java:197)
at com.mycompany.pt.transactionFiles.TransactionFileDataflow$1.processLine(TransactionFileDataflow.java:411)
at com.mycompany.pt.transactionFiles.TransactionFileDataflow$1.processElement(TransactionFileDataflow.java:336)
at com.mycompany.pt.transactionFiles.TransactionFileDataflow$1$DoFnInvoker.invokeProcessElement(Unknown Source)
`
Does anyone have any experience using Spanner like this within a ParDo?
I'm not a spanner expert, but maybe I can help:
You should use #Setup/#Teardown to connect & disconnect from spanner. #{Start,Finish}Bundle gets called multiple times over the lifetime of a worker. See here for more details: https://beam.apache.org/documentation/execution-model/#bundling-and-persistence
Does your processElement method ever emit an element using
c.output(...)? If not, beam will think your pipeline is stuck

DynamoDb not able to save map data(java)

I am trying to save data into the dynamoDb but that data contains some Map attributes as well.But I am getting error while saving that data. Following is my domain class which I am using for saving data from request:
#DynamoDBTable(tableName = "ottMiddleware_rails")
public class RailsCmsDomain {
#DynamoDBHashKey(attributeName = "railId")
private String railId;
#DynamoDBTyped
#DynamoDBAttribute(attributeName = "railLogic")
private Map<String, Object> railLogic;
#DynamoDBAttribute(attributeName = "railSourceType")
private String railSourceType;
#DynamoDBAttribute(attributeName = "railTitle")
private RailCmsTitleDomain railTitle;
#DynamoDBTyped
#DynamoDBAttribute(attributeName = "restrictions")
private Map<String, Object> restrictions;
I am giving the following request:
{
"railId": "railOne",
"railLogic": {
"programType": 1,
"railSourceUrl": "http://myUrl"
},
"railSourceType": "myRail",
"railTitle": {
"tam": "Raan Phan",
"def": "சிறப்பு கட்டமைப்பு"
},
"restrictions": {
"clients": [
"abc",
"xyz"
],
"periodStart": 1506572217
}
}
I am using following code to save my data into the dynamoDb
public Boolean saveUpdateRailsDetails(RailsCmsDomain railsDomain) {
DynamoDBUtil dynamoDBUtil = new DynamoDBUtil();
AmazonDynamoDB dynamoDBClient = dynamoDBUtil.getDynamoDBClient();
DynamoDBMapper mapper = new DynamoDBMapper(dynamoDBClient);
mapper.save(railsDomain);
return true;
}
Please suggest how can I save map into dynamoDb. I am taking data as map because in later stages there are chances more data can be added to those attributes which are map and that data can be of any data type. I am getting following error:
errorMessage": "not supported; requires #DynamoDBTyped or
#DynamoDBTypeConverted"
"errorType":
"com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMappingException"
I would like to post this answer as an option though it is not going to store the data as map in DynamoDB. However, it will resolve the error.
You can define the restrictions attribute as mentioned below. It will store the data as JSON string in DynamoDB.
#DynamoDBTypeConvertedJson
private Map<String, Object> restrictions;
Drawbacks:-
When you would like to update the restrictions attribute, you need to get the current value from database, update it and save the data in database.
DynamoDB data:-
restrictions attribute saved as JSON string.
When I needed to store map in dynamodb I just annotated the getter method of the map with #DynamoDBTypeConverted(converter = MapConverter.class) and in the converter just serialized the map into a json string, this way you can retrieve the map and everything but are not going to be able to use it in queryExpressions if you need.
So in case you want to be able to query the objects in the map maybe consider annotating the object you are using in your map with #DynamoDBDocument this way it's going to be automatically serialized by dynamodb as a document and use Set instead of map into your parent object.
The problem is that the DynamoDB Mapper doesn't know how to marshall/unmarshall the Object in this line:
private Map<String, Object> railLogic;
If you don't want to change Object to String, then use the DynamoDBTypeConverted annotation to provide a custom marshaller for Map<String, Object> which you can then (probably) reuse for restrictions. Relevant documentation here.

DynamoDb scan not working properly

I am trying to fetch items from the dynamoDb. However when I fetch single item using partiton key everything works fine but when I try to fetch all the items in dynamo db using scan I encouter an error. Following is the code what I am trying to do:
public List<PageCmsDomain> getAllPages() {
DynamoDBUtil dynamoDBUtil = new DynamoDBUtil();
AmazonDynamoDB dynamoDBClient = dynamoDBUtil.getDynamoDBClient();
DynamoDBMapper mapper = new DynamoDBMapper(dynamoDBClient);
List<PageCmsDomain> scanResult = mapper.scan(PageCmsDomain.class, new
DynamoDBScanExpression());
return scanResult;
Following error comes when I execute this code:
com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMappingException: PageCmsDomain[restrictions]; could not unconvert attribute
at com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapperTableModel.unconvert(DynamoDBMapperTableModel.java:271)
at com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper.privateMarshallIntoObject(DynamoDBMapper.java:456)
at com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper.marshallIntoObjects(DynamoDBMapper.java:484)
at com.amazonaws.services.dynamodbv2.datamodeling.PaginatedScanList.<init>(PaginatedScanList.java:64)
at com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper.scan(DynamoDBMapper.java:1458)
at com.amazonaws.services.dynamodbv2.datamodeling.AbstractDynamoDBMapper.scan(AbstractDynamoDBMapper.java:216)
at com.astro.ott.dynamodb.cmsapi.impl.PageCmsDaoImpl.getAllPages(PageCmsDaoImpl.java:74)
at com.astro.ott.cmsapi.impl.PageCmsGetServiceImpl.getPages(PageCmsGetServiceImpl.java:41)
at com.astro.ott.cms_api.cms_api.PageCmsTestCase.get(PageCmsTestCase.java:81)
Caused by: com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMappingException: not supported; requires #DynamoDBTyped or #DynamoDBTypeConverted
at com.amazonaws.services.dynamodbv2.datamodeling.StandardModelFactories$Rules$NotSupported.get(StandardModelFactories.java:660)
at com.amazonaws.services.dynamodbv2.datamodeling.StandardModelFactories$Rules$NotSupported.get(StandardModelFactories.java:650)
at com.amazonaws.services.dynamodbv2.datamodeling.StandardModelFactories$AbstractRule.unconvert(StandardModelFactories.java:714)
at com.amazonaws.services.dynamodbv2.datamodeling.StandardModelFactories$AbstractRule.unconvert(StandardModelFactories.java:691)
at com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTypeConverter$DelegateConverter.unconvert(DynamoDBTypeConverter.java:109)
at com.amazonaws.services.dynamodbv2.datamodeling.StandardTypeConverters$Vector$ToMap.unconvert(StandardTypeConverters.java:433)
at com.amazonaws.services.dynamodbv2.datamodeling.StandardTypeConverters$Vector$ToMap$1.unconvert(StandardTypeConverters.java:417)
at com.amazonaws.services.dynamodbv2.datamodeling.StandardTypeConverters$Vector$ToMap$1.unconvert(StandardTypeConverters.java:410)
at com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTypeConverter$DelegateConverter.unconvert(DynamoDBTypeConverter.java:109)
at com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTypeConverter$NullSafeConverter.unconvert(DynamoDBTypeConverter.java:128)
at com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTypeConverter$ExtendedConverter.unconvert(DynamoDBTypeConverter.java:88)
at com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapperFieldModel.unconvert(DynamoDBMapperFieldModel.java:146)
at com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapperFieldModel.unconvertAndSet(DynamoDBMapperFieldModel.java:164)
at com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapperTableModel.unconvert(DynamoDBMapperTableModel.java:267)
... 32
And I am using same PageCmsDomain while fetching single item also.
Specified code will work only if restrictions field is null or empty map. Otherwise it will throw exception as DynamoDB does not allow to save or fetch objects that have field with Object type (even as a generic type, in this case - map's value type Object), instead it should have specific type (e.g. String, Integer, Map<String, Integer>).
If restrictions map's value type can't be changed to specific one,
you can fix that by adding DynamoDBTypeConvertedJson annotation to store serialized value of restrictions field:
#DynamoDBAttribute(attributeName = "restrictions")
#DynamoDBTypeConvertedJson
private Map<String, Object> restrictions;
Another option is to specify your own custom converter class and annotate restrictions field with
#DynamoDBTypeConverted(converter = CustomConverter.class)

Increment Number Property in AWS DynamoDB

How do I increment a number in AWS Dynamodb?
The guide says when saving an item to simply resave it:
http://docs.aws.amazon.com/mobile/sdkforios/developerguide/dynamodb_om.html
However I am trying to use a counter where many users may be updating at the same time.
Other documentation has told me to use and UpdateItem operation but I cannot find a good example to do so.
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.Modifying.html
However, I cannot find a method to implement the expression. In the future I will be adding values to arrays and maps. Will this be the same? My code is in Obj C
Currently my code looks like:
AWSDynamoDBUpdateItemInput *updateItemInput = [AWSDynamoDBUpdateItemInput new];
updateItemInput.tableName = #"TableName";
updateItemInput.key = #{
UniqueItemKey:#"KeyValue"
};
updateItemInput.updateExpression = #"SET counter = counter + :val";
updateItemInput.expressionAttributeValues =#{
#":val":#1
};
It looks like you're missing the last bit of code that actually makes the update item request:
AWSDynamoDB *dynamoDB = [AWSDynamoDB defaultDynamoDB];
[[dynamoDB updateItem:updateItemInput]
continueWithBlock:^id(AWSTask *task) {
if (task.error) {
NSLog(#"The request failed. Error: [%#]", task.error);
}
if (task.exception) {
NSLog(#"The request failed. Exception: [%#]", task.exception);
}
if (task.result) {
//Do something with result.
}
return nil;
}];
In DynamoDB if you want to increment the value of the any propertie/field you can use the UpdateItemRequest with action option ADD. I used in android this method would update the existing value of the field. Let me share the code snippet. You can use any actions such like add,delete,put etc.
.....
AttributeValue viewcount = new AttributeValue().withS("100");
AttributeValueUpdate attributeValueUpdate = new AttributeValueUpdate().withAction(AttributeAction.ADD).withValue(viewcount);
updateItems.put(UploadVideoData.FIELD_VIEW_COUNT, attributeValueUpdate);
UpdateItemRequest updateItemRequest = new UpdateItemRequest().withTableName(UploadVideoData.TABLE_NAME)
.withKey(primaryKey).withAttributeUpdates(updateItems);
UpdateItemResult updateItemResult = amazonDynamoDBClient.updateItem(updateItemRequest);
....
You can see the above code will add 100 count into the existing value of that field.
This code is for android but the technique would remain the same.
Thank you.