How to update item by Composite Primary Key in Dynamodb - amazon-web-services

I have a table called friends:
Friend 1 | Friend 2 | Status
Friend 1 is my HASH attribute and Friend 2 is my range attribute.
I would like to update an item's staus attribute where friend 1 = 'Bob' and friend 2 = 'Joe'. Reading through the documentation on http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/JavaDocumentAPICRUDExample.html I can only see how to update an item by 1 key, how do I include the other key?

Here you go:
DynamoDBQueryExpression<Reply> queryExpression = new DynamoDBQueryExpression<Reply>()
.withKeyConditionExpression("Id = :val1 and ReplyDateTime > :val2")
.withExpressionAttributeValues(
...
where Id is the Hash Key and ReplyDateTime is the Range Key.
Reference:
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBMapper.QueryScanExample.html

I'm writing example where you can make update of multiple item in single table. I have primary key as id and range key as Datetime.
Actually there is no feature available in dynamodb so what i'm doing here is first query all the variable with hash key and range key of which i want to make update. Once all data are stored in List then loading data with it's hash key and rangekey and changing or updating field using set and save it.
Since I'm editing in hash key so, hash key original will be there we need to delete it. If you need to update in next attribute no need. I haven't added deleting code write yourself. You can query if you have confusion your entry with hash key will be still and new entry with new hash key will be added.
Code is below:
public static void main(String[] args) {
AmazonDynamoDBClient client = new AmazonDynamoDBClient();
DynamoDBMapper mapper = new DynamoDBMapper(client);
client.setEndpoint("http://localhost:8000/");
String fromDate = "2016-01-13";
String toDate = "2016-02-05";
User user = new User();
user.setId("YourHashKey");
LocalDate frmdate = LocalDate.parse(fromDate, DateTimeFormatter.ISO_LOCAL_DATE);
LocalDate todate = LocalDate.parse(toDate, DateTimeFormatter.ISO_LOCAL_DATE);
LocalDateTime startfrm = frmdate.atStartOfDay();
LocalDateTime endto = todate.atTime(23, 59, 59);
Condition rangeCondition = new Condition().withComparisonOperator(ComparisonOperator.BETWEEN.toString()).withAttributeValueList(new AttributeValue().withS(startfrm.toString()), new AttributeValue().withS(endto.toString()));
DynamoDBQueryExpression<User> queryExpression = new DynamoDBQueryExpression<User>().withHashKeyValues(user).withRangeKeyCondition("DATETIME", rangeCondition);
List<User> latestReplies = mapper.query(User.class, queryExpression);
for (User in : latestReplies) {
System.out.println(" Hashid: " + in.getId() + " DateTime: " + in.getDATETIME() + "location:" + in.getLOCID());
User ma = mapper.load(User.class, in.getId(), in.getDATETIME());
ma.setLOCID("Ohelig");
mapper.save(ma);
}
}

Related

emr-dynamodb-connector don't save if primary key is present in dynamodb

We are using Spark job with emr-dynamodb-connector to load the data from S3 files into Dyanamodb.
https://github.com/awslabs/emr-dynamodb-connector
But if document is already present in dynamodb, my code is replacing it.
Is there a way to avoid updating existing records (based on id) if they are present in Dynamodb. If id is present in dynamodb, i simply don't want to update it, just skip that id and write rest of records. Code i am using is
JobConf ddbConf = new JobConf(spark.sparkContext().hadoopConfiguration());
ddbConf.set("dynamodb.output.tableName", tableName);
ddbConf.set("dynamodb.throughput.write.percent", "50");
ddbConf.set("mapred.input.format.class", "org.apache.hadoop.dynamodb.read.DynamoDBInputFormat");
ddbConf.set("mapred.output.format.class", "org.apache.hadoop.dynamodb.write.DynamoDBOutputFormat");
JavaPairRDD<Text, DynamoDBItemWritable> ddbInsertFormattedRDD = finalDatasetToBeSaved.toJavaRDD().mapToPair(new PairFunction<Row, Text, DynamoDBItemWritable>() {
#Override
public Tuple2<Text, DynamoDBItemWritable> call(Row row) throws Exception {
Map<String, AttributeValue> ddbMap = new HashMap<String, AttributeValue>();
for (int i = 0 ; i <= schemaDdb.length - 1; i++) {
Object value = row.get(i);
if (value != null) {
AttributeValue att = new AttributeValue();
if(schemaDdb[i]._2.toString().equalsIgnoreCase("IntegerType")){
att.setN(value.toString());
}else{
att.setS(value.toString());
}
ddbMap.put((String)schemaDdb[i]._1, att);
}
}
DynamoDBItemWritable item = new DynamoDBItemWritable();
item.setItem(ddbMap);
return new Tuple2<Text, DynamoDBItemWritable>(new Text(""), item);
}
});
ddbInsertFormattedRDD.saveAsHadoopDataset(ddbConf);
By saying Is there a way to avoid updating existing records (based on id) if they are already present, Do you want to add another document instead of replacing/updating it?
If yes, I am afraid it wont be possible with primary key, since that should be unique and distinguishes it from other. You need to make a key non-primary in order to do this.
If you want to ignore the insertion (if item exists), you can use condition-expression attribute_not_exists(your-key) as defined in the documentation

Save Expression on Encrypted attribute in DynamoDB

I'm using save expression on an encrypted attribute named transactionAmount while updating data in dynamo DB. However the update query is failing with ConditionalCheckFailedException. The data is encrypted on client side during initial persistence in dynamodb in way same as described here. Following is the code:
Data Transfer Object:
public final class SampleDTO {
#DynamoDBHashKey(attributeName = CommonDynamoDBSchemaConstants.UNIQUE_KEY)
#Getter(onMethod = #__({ #DoNotTouch }))
private String uniqueKey;
#DynamoDBAttribute(attributeName = CommonDynamoDBSchemaConstants.EVENT_RUNNING_TIME_EPOCH)
#Getter(onMethod = #__({ #DoNotTouch }))
private Long eventRunningTimeInEpoch;
#DynamoDBAttribute(attributeName = CommonDynamoDBSchemaConstants.INSTRUMENT_TYPE)
#DynamoDBTypeConverted(converter = InstrumentTypeConverter.class)
#Getter(onMethod = #__({ #DoNotTouch }))
private InstrumentType instrumentType;
#DynamoDBAttribute(attributeName = CommonDynamoDBSchemaConstants.TRANSACTION_AMOUNT)
private String transactionAmount;
}
Data Access Code:
// fetches data from dynamoDB based on unique key passed to it.
SampleDTO sampleDTO = getSampleDTO("testLedgerUniqueKey");
sampleDTO.setInstrumentType(InstrumentType.MACHINE);
DynamoDBSaveExpression saveExpression = new DynamoDBSaveExpression();
Map<String, ExpectedAttributeValue> expressionAttributeValues =
new HashMap<String, ExpectedAttributeValue>();
expressionAttributeValues.put(
CommonDynamoDBSchemaConstants.LEDGER_UNIQUE_KEY,
new ExpectedAttributeValue(true)
.withValue(new AttributeValue(sampleDTO.getLedgerUniqueKey())));
expressionAttributeValues.put(
CommonDynamoDBSchemaConstants.TRANSACTION_AMOUNT,
new ExpectedAttributeValue(true).withValue(
new AttributeValue(sampleDTO.getTransactionAmount())));
saveExpression.setExpected(expressionAttributeValues);
saveExpression.setConditionalOperator(ConditionalOperator.AND);
dynamoDBMapper.save(sampleDTO, saveExpression, null /*dynamoDBMapperConfig*/);
ConditionalCheckFailedException:
You are trying to update a record that does not exist with your query condition. Please verify your query condition to make sure your query returns a record.
Reference:
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Programming.Errors.html#Programming.Errors.MessagesAndCodes
You specified a condition that evaluated to false. For example, you
might have tried to perform a conditional update on an item, but the
actual value of the attribute did not match the expected value in the
condition.
Hope it helps.

Dynamodb pagination of 10 results

I have a message table which I would like to get the last 10 messages for that user and if they click more it would retrieve another 10 until there were no more message left. I could not see how to do this, so far I am able to get message based on time, but this is not exactly what I want. Reading through the documentation I can see it a method called LastEvaluatedKey, but where and how do I use this I can't find a working example of this. This is my code for time:
long startDateMilli = (new Date()).getTime() - (15L*24L*60L*60L*1000L);
long endDateMilli = (new Date()).getTime() - (5L*24L*60L*60L*1000L);
java.text.SimpleDateFormat df = new java.text.SimpleDateFormat(
"yyyy-MM-dd'T'HH:mm:ss.SSS'Z'");
String startDate = df.format(startDateMilli);
String endDate = df.format(endDateMilli);
QuerySpec spec = new QuerySpec()
.withProjectionExpression("to,fr,sta,cr")
.withKeyConditionExpression("to = :v_to and cr between :v_start_dt and :v_end_dt")
.withValueMap(new ValueMap()
.withString(":v_id", username)
.withString(":v_start_dt", startDate)
.withString(":v_end_dt", endDate));
ItemCollection<QueryOutcome> items = table.query(spec);
System.out.println("\nfindRepliesPostedWithinTimePeriod results:");
Iterator<Item> iterator = items.iterator();
while (iterator.hasNext()) {
System.out.println(iterator.next().toJSONPretty());
}
How can I modify this to eliminate the time based pagination and use instead a last 10 messages type of pagination?

List of instances with minimal date of their group

I'm working on a Java project, using Hibernate to administrate data on a SQL database.
I try to fetch a list of instances from the Database, that have a minimal timestamp of the group they share. The group is modeled by a container.
Here is a minimal model sketch:
#Entity
#Table(name = "object")
public class Object implements Serializable{
#Id
#GeneratedValue(strategy = GenerationType.Auto)
long obj_id;
#Column(name = "time_stamp", nullable = false)
Date timestamp;
#ManyToOne(fetch = FetchType.EAGER)
#JoinColumn(name = "container_id", nullable = false)
Container con;
}
#Entity
#Table(name = "container")
public class Container{
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
long con_id;
#OneToMany(mappedBy = "container")
List<object> obj_list;
}
So there are some objects with a timestamp and containers that group these objects.
For example, there are two containers, con_a and con_b:
Container con_a:
con_id = 1
obj_list = {obj_a, obj_b}
Container con_b:
con_id = 2
obj_list = {obj_c}
And three objects, obj_a, obj_b, obj_c:
Object obj_a
obj_id = 1
timestamp = 10
con = con_a
Object obj_b
obj_id = 2
timestamp = 20
con = con_a
Object obj_c
obj_id = 3
timestamp = 30
con = con_b
The desired List in this example would look like:
List<Object> = {obj_a, obj_c}
I seem to move in a circle, as I do not even know where to "start" the query:
Criteria crit = session.createCriteria(Container.class). ...
or
Criteria crit = session.createCriteria(Object.class). ...
It seems both possible for me, but i just have no idea how to go on from any of those 2 possibilities.
Update [2014.07.11, 14:19]:
I tried and started the query with the Object class and used a Subquery:
Session session = getSession();
Transaction transaction = session.beginTransaction();
DetachedCriteria IdListOfGroupMinimum = DetachedCriteria.forClass(Object.class, "obj")
IdListOfGroupMinimum.createAlias("con.id", "containerId")
.setProjection(
.Projections.projectionList()
.add(Projections.property("obj.id"))
.add(Projections.min("obj.timestamp"))
.add(Projections.groupProperty("containerId")))
.setProjection(Projection.property("obj.id"));
Criteria objects = session.createCriteria(object.class, "obj")
objects.add(Subqueries.in("obj.id", IdListOfGroupMinimum));
List<Object> = objects.list();
But I received the following error:
javax.servlet.ServletException: org.hibernate.QueryException: not an association: id
I tried to do this:
SELECT * from Object
WHERE id IN (
SELECT obj.id
FROM Object obj
INNER JOIN (
SELECT obj.containerID, MIN(obj.timestamp) AS minimum
FROM Object obj
GROUP BY obj.containerID) subquery
ON obj.containerID = subquery.containerID
WHERE obj.timestamp = subquery.minimum
)
I found a solution for my problem which is probably not the most elegant one, but it works.
Mainly I used the SQL-Query that I already posted above:
Session session = getSession();
Transaction transaction = session.beginTransaction();
//This query fetches the IDs of the smallest objects in each group with
//regard to the timestamp
Query q = session.createSQLQuery(
"SELECT obj.id FROM Object obj "
+ "INNER JOIN ( "
+ "SELECT obj.containerID, MIN(obj.timestamp) AS minimum "
+ "FROM Object obj "
+ "GROUP BY obj.containerID) subquery "
+ "ON obj.containerID = subquery.containerID "
+ "WHERE obj.timestamp = subquery.minimum "
);
//This tells Hibernate that the result are values of type Long
q.addScalar("id", LongType.INSTANCE)
//Creates a list of the found IDs
#SuppressWarnings("unchecked")
List<Long> ids = q.list();
//Fetches all object with those IDs...
Criteria smallestOfEachGroup = session.createCriteria(Object.class)
.add(Restrictions.in("id", ids);
//...and saves them in a list.
#SuppressWarnings("unchecked")
List<Object> desiredList = smallestOfEachGroup.list()
try{
transaction.commit();
} catch(HibernateException e) {
transaction.rollback();
}
As all my sketches are not the real code, so there might be still naming errors.
Anyway, I hope this helps someone.
I still would be pleased by any more elegant solution.
Update [2014.07.20, 18:50]:
I found a solution that uses Hibernate Criteria exclusively :)
Session session = getSession();
Transaction transaction = session.beginTransaction();
//This subquery fetches the minimal timestamp of a container.
DetachedCriteria minOfGroup = DetachedCriteria.forClass(Object.class);
minOfGroup.add(Restrictions.eqProperty("con.con_id", "outerObject.con.con_id")
.setProjection(Projections.min("timestamp"));
//This subquery fetches the IDs of all Objects, whose timestamp is minimal
//in their container.
DetachedCriteria groupwiseMin = DetachedCriteria.forClass(Object.class, "outerObject");
groupwiseMin.add(Subqueries.propertyEq("timestamp", minOfGroup));
.setProjections(Projections.id())
//This subquery fetches all Objects whose IDs are fetched by the groupwiseMin
//query
Criteria groupwiseMinObjects = session.createCriteria(Object.class);
groupwiseMinObjects.add(Subqueries.propertyIn("obj_id", groupwiseMin));
List<Object> desiredObjects = groupwiseMinObjects.list();
try{
transaction.commit();
} catch(HibernateException e) {
transaction.rollback();
}
I think you can make this query even shorter, if you remove the groupwiseMinObjects query above replace the groupwiseMin query by:
Criteria anotherGroupWiseMinObjects = session.createCriteria(Object.class, "outerObject");
anotherGroupwiseMinObjects.add(Subqueries.propertyEq("timestamp", minOfGroup));
But I did not test that.
In my original project I use several subqueries that converge in a single query.
That means after some subqueries, there is a final query like:
Criteria finalQuery = session.createCriteria(Object.class);
finalQuery.add(Subqueries. (...) )
(...)
.add(Subqueries. (...) );

CF10 new query()

I am creating a query inside a function. Everything works fine until this line:
ulist = new query();
Then I get the error:
Could not find the ColdFusion component or interface query.
Code:
//GET USERS LISTS
remote query function getUserLists(userid) {
//CONFIGURE twitter4j
init();
//DEFINE USER LIST QUERY
var userLists = querynew("id, name, member_count", "Integer, VarChar, Integer");
//GET THE USER LISTS
getLists = t4j.getUserLists(#arguments.userid#);
//BUILD THE USER LIST QUERY
for (i=1;i LTE ArrayLen(getLists);i=i+1) {
newRecord = queryAddRow(userLists);
newRecord = querySetCell(userLists, "id", getLists[i].getId());
newRecord = querySetCell(userLists, "name", getLists[i].getName());
newRecord = querySetCell(userLists, "member_count", getLists[i].getMemberCount());
}
//SORT THE USER LIST BY NAME
ulist = new query();
ulist.setDBType("query");
ulist.setAttributes(sourceQuery=userLists);
ulist.setSQL("select * from sourceQuery order by name");
userListsSorted = ulist.execute().getresult();
//RETURN THE SORTED USER LIST QUERY
return userListsSorted;
}
As per Twitter, make sure you have a custom tag path pointing to [instance]/customtags - which should be there by default. You could use a mapping, pointing to one of the subdirectories within that [instance]/customtags directory, eg: /coldfusion pointing to [instance]\CustomTags\com\adobe\coldfusion, then use:
ulist = new coldfusion.query();
// etc
I'd just use the custom tag directory approach though.
Try using the full path:
ulist = new com.adobe.coldfusion.query()