I am reading https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/TTL.html to add a TTL field on a Dynamodb table. But I can't find how it supports a nested field as TTL field. For example,
id: xxx
user: { firstName: '', lastName:'', age: ''}
in above example, how can I use user -> age as ttl field?
First, a string can't be used for TTL. If you try and do that, it will be ignored, as per documentation:
The TTL attribute’s value must be a Number data type. For example, if you specify for a table to use the attribute name expdate as the TTL attribute, but the attribute on an item is a String data type, the TTL processes ignore the item.
Also from the same documentation page:
The TTL attribute’s value must be a timestamp in Unix epoch time format in seconds. If you use any other format, the TTL processes ignore the item. For example, if you set the value of the attribute to 1645119622, that is Thursday, February 17, 2022 17:40:22 (GMT), the item will be expired after that time.
I am mentioning this because I am getting an impression you want to use the user's age in some way as a TTL, and that is not a timestamp value.
And also, as #jarmod has said in the comment, it has to be a top-level attribute. You have to extract the value and add it to the TTL column, whichever it is, in a Number format timestamp value in seconds in the future.
Related
I have a dynamodb table with following GSI:
partition key: scheduled_date which is a date string yyyy-mm-dd HH:MM:SS
range key: task_id which is an uuid
I would like to query for all items whose scheduled_date falls in a date, i.e. its prefix matches a string yyyy-mm-dd.
Is it possible without performing scan?
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/LegacyConditionalParameters.KeyConditions.html
You must provide the index partition key name and value as an EQ
condition.
In your case, you could consider using yyyy-mm-dd (or yyyymmdd) as the partition key to get all of the items that have that scheduled date.
You could keep task_id as the Range key OR you could use a prefix like HH:MM:SS:task_id. That way the tasks for a particular day would come back sorted by time. And if you really needed to you, could query them by time range.
There is also the alternative of using Global Secondary Indexes that can be utilized in a similar manner.
I've following dynamoDB table
user_id
date
game
user1
2021-12-06 14:36:46
game1
user1
2021-12-06 15:36:46
game1
user1
2021-12-07 11:36:46
game2
user1
2021-12-07 12:36:46
game2
partition key: user_id
sort key: date
I want to Query the latest entry of user for game game1
(Which is the second item from table with date 2021-12-06 15:36:46). I can achieve this from code as follows;
expr, _ := expression.NewBuilder().
WithKeyCondition(expression.Key("user_id").Equal(expression.Value("user1"))).
WithFilter(expression.Name("game").Equal(expression.Value("game1"))).
Build()
var queryInput = &dynamodb.QueryInput{
KeyConditionExpression: expr.KeyCondition(),
ExpressionAttributeNames: expr.Names(),
ExpressionAttributeValues: expr.Values(),
FilterExpression: expr.Filter(),
ScanIndexForward: aws.Bool(false),
TableName: aws.String(table),
}
This returns me all items of user user1 for game game1. Problem occurs when I apply limit=1 Limit: aws.Int64(1) in QueryInput, it returns nothing. Could someone explain why is that so ?
When I change Limit: aws.Int64(4) (total number of items in table), only then the query returns single expected item. How is this limit working ?
Do I need to use game as GSI ?
The limit on a DDB parameter is applied before your filter expressions.
Essentially with a limit of 1, it retrieves 1 record, then applies the filters and returns you the items that match (0).
See https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html#Query.Limit for more details, copied in case link breaks
Limiting the Number of Items in the Result Set
The Query operation allows you to limit the number of items that it reads. To do this, set the Limit parameter to the maximum number of items that you want.
For example, suppose that you Query a table, with a Limit value of 6, and without a filter expression. The Query result contains the first six items from the table that match the key condition expression from the request.
Now suppose that you add a filter expression to the Query. In this case, DynamoDB reads up to six items, and then returns only those that match the filter expression. The final Query result contains six items or fewer, even if more items would have matched the filter expression if DynamoDB had kept reading more items.
I have a dynamo-db table with following schema
{
"id": String [hash key]
"type": String [range key]
}
I have a usecase where I need to fetch last 3 rows for a given id when type is unknown.
Your items need a timestamp attribute. Without that they can’t be sorted out filtered by time. Once you have that, you can define a local secondary index with the id as partition key and the timestamp as the sort key. You can then get the top three items from the index.
Find more information about DynamoDb’s Local Secondary Index here.
Add a field to store the timestamp to the schema
Use query to fetch all the records for the given key
Query always returns records sorted by range key, you cannot set a sort order (without changing table's schema), so, sort the records by timestamp in your code
Get top 3 records
If you have a lot of records, use filter expressions to drop extra results. E.g. if you know that latest records will always have a timestamp not older than a hour (day, week or so) you could filter older records.
I have setup a TTL attribute in my dynamoDB table. when i push records in a get the current date (using js sdk in node) and add a value to it (like 5000). It is my understanding that when that date is reached aws will purge the record but only within 48 hours. during that time the record could be returned as the result of a query.
I want to filter out the expired items so that if they are expired but not deleted they won't be returned as part of the query.
here is what i am using to try to do that:
var epoch = Math.floor(Date.now() / 1000);
console.log("ttl epoch is ", epoch);
var queryTTLParams = {
TableName : table,
KeyConditionExpression: "id = :idval",
ExpressionAttributeNames:{
"#theTTL": "TTL"
},
FilterExpression: "#theTTL < :ttl",
ExpressionAttributeValues: {
":idval": {S: "1234"},
":ttl": {S: epoch.toString()}
}
};
i do not get any results. I believe the issue has to do with the TTL attribute being a string and me trying to do a < on it. But i didn't get to decide on the datatype for the TTL field - aws did that for me.
How can i remedy this?
According to the Enabling Time to Live AWS documentation, the TTL should be set to a Number attribute:
TTL is a mechanism to set a specific timestamp for expiring items from your table. The timestamp should be expressed as an attribute on the items in the table. The attribute should be a Number data type containing time in epoch format. Once the timestamp expires, the corresponding item is deleted from the table in the background.
You probably just need to create a new Number attribute and set the TTL attribute to that one.
Apologies, I am completely new to Django. My question is that I have 20 records in my database table and suppose 10 record is of same ID and I want to fetch last inserted record for that id I have date column in my table. How can I do that?
last_obj = YourModel.objects.last()
But generally, you can't create > 1 objects with same id, if you didn't specified your own id field to replace built-in. And even then, it's a bad idea.