My DynamoDB table currently has the PK, SK and a GSI called "EmailIndex".
The idea behind this is that when users create their account, the username will be set as the PK and the email would be set as the EmailIndex GSI.
This should let me allow the user to login with either the username or the email.
Is it possible to reliably ensure there is no duplicated EmailIndex record? If for example, two different people enter the exact same email at the exact same time with different usernames, how can I ensure both users don't end up creating a record with the same GSI?
Right now I'm under the assumption that this isn't actually possible with DynamoDB. In which case, what would be the acceptable and recommended approach of allowing username or email logins instead of mandating one or the other?
There isn't a way to guarantee unique values on a GSI. What I've done in the past to solve this is to have two records for the user, one with the data you have today, and one that is keyed on the email address. When doing a Put operation you can use a transaction to be sure both records succeed. You can still put the email in the GSI on the main record and use that for querying, or use the second record instead, and duplicate the data (which is what the GSI would do if you have the projection set to ALL).
Related
ex.
Let's say I'm trying to create an eCommerce platform with multiple sellers and I create a Table called Orders. The partition key will be storeID and the sort key will be orderNumber.
Stores can call API.get('Orders', {storeID}), which will return all the items with the partition key of storeID.
My application uses Amazon Cognito and each user is assigned a username which is a uuid. My question is can I use the uuid as the storeID in my DynamoDB table? The key assumption is that attackers won't be able to guess the uuid.
You should always validate access to resources server-side.
Not being able to guess a uuid isn't a safe assumption. Since you are using Amazon Cognito, there should be a way in your server code to get the logged-in user (the uuid). When you are making a query to DynamoDB, you shouldn't rely on a uuid passed by an HTTP query (client-side), but use instead the uuid value of the logged-in user.
The uuid could be used in a Global Secondary Index, so that you can quickly query the orders of a user.
you can use UUID to store the same users data in dynamo as well. But make sure while fetching the details in dynamo what detail an API is actually needed like whether a store API should request only orders and not users so to differentiate those stuffs just add an attribute called "entity_type" which will have
order
user
store
other resources
So while fetching add this entity_type as well one of the where condition to filter out rest of the unwanted stuffs.
on a single table pattern on dynamodb, when you have a many to many relationship, how would you make an update
lets say a user-account table, consisting of a userId and accountId as the partition key & sort key, and a gsi of the reverse, a user can have many accounts and an account can have many users, what happens when the information on an account updates
would you go through and loop through all accounts and update each individual record?
For example, if a user updates an account information, then all users linked to that account would need the updated information now. In SQL it would be easy as the accounts would be linked via a foreign key, but in nosql, each row contains all the information about the account, so what would be the best approach to maintain consistent data for the updated account accross all users?
I have an app where a user must be logged in to post an advertisement visible to others. How can I associate that person's unique id to that posting? Is there a way to fetch their attributes once they are signed in?
edit: each user submits a form to add the posting to the db.
I have the practice of putting triggers on every remotely important table.
Each of these tables includes 4 fields, date of insert, user who inserted, date of last change, user of last change.
Then the trigger just fills these fields with SYSDATE and APP_USER.
This way we get to see who inserted each data, and if it was later changed, we also see that.
For more important tables you should also have history, either the built in history, or a table into which each change is logged.
Each user will have a unique value in the :APP_USER substitution string.
I store user accounts in DynamoDB:
{
email: 'user1#xxx.com',
expires: 1548807053247,
}
My hash key is the email field.
I want to add a daily cron job which will send a reminder email for all accounts about to expire (in the next 14 days).
For that, I need to query on expires field alone - without using the hash key.
I assume I need to define a secondary index on this field (probably global and not local?), but I'm not sure on how to write the proper query for it.
I'm using AWS.DynamoDB.DocumentClient for accessing the table, thanks in advance!
Just specify the IndexName in addition to the TableName when you call the Query API. (Docs.) The rest is the same as if you were querying the table.
Im having the same problem very often
what I do is , add another attribute named "all" with value of 1
(you can use any key/value )
and then create a GSI
partition: all
sort: expires
to optimise a bit you could add this attribute only to active users
or you could add an attribute active Number and use it as partition key for the GSI
this is a very inefficient partition key distribution
since all items belong to one partition
but I found no other way around
I'd be happy to hear another solution
I'm building a DynamoDB app that will eventually serve a large number (millions) of users. Currently the app's item schema is simple:
{
userId: "08074c7e0c0a4453b3c723685021d0b6", // partition key
email: "foo#foo.com",
... other attributes ...
}
When a new user signs up, or if a user wants to find another user by email address, we'll need to look up users by email instead of by userId. With the current schema that's easy: just use a global secondary index with email as the Partition Key.
But we want to enable multiple email addresses per user, and the DynamoDB Query operation doesn't support a List-typed KeyConditionExpression. So I'm weighing several options to avoid an expensive Scan operation every time a user signs up or wants to find another user by email address.
Below is what I'm planning to change to enable additional emails per user. Is this a good approach? Is there a better option?
Add a sort key column (e.g. itemTypeAndIndex) to allow multiple items per userId.
{
userId: "08074c7e0c0a4453b3c723685021d0b6", // partition key
itemTypeAndIndex: "main", // sort key
email: "foo#foo.com",
... other attributes ...
}
If the user adds a second, third, etc. email, then add a new item for each email, like this:
{
userId: "08074c7e0c0a4453b3c723685021d0b6", // partition key
itemTypeAndIndex: "Email-2", // sort key
email: "bar#bar.com"
// no more attributes
}
The same global secondary index (with email as the Partition Key) can still be used to find both primary and non-primary email addresses.
If a user wants to change their primary email address, we'd swap the email values in the "primary" and "non-primary" items. (Now that DynamoDB supports transactions, doing this will be safer than before!)
If we need to delete a user, we'd have to delete all the items for that userId. If we need to merge two users then we'd have to merge all items for that userId.
The same approach (new items with same userId but different sort keys) could be used for other 1-user-has-many-values data that needs to be Query-able
Is this a good way to do it? Is there a better way?
Justin, for searching on attributes I would strongly advise not to use DynamoDB. I am not saying, you can't achieve this. However, I see a few problems that will eventually come in your path if you will go this root.
Using sort-key on email-id will result in creating duplicate records for the same user i.e. if a user has registered 5 email, that implies 5 records in your table with the same schema and attribute except email-id attribute.
What if a new use-case comes in the future, where now you also want to search for a user based on some other attribute(for example cell phone number, assuming a user may have more then one cell phone number)
DynamoDB has a hard limit of the number of secondary indexes you can create for a table i.e. 5.
Thus with increasing use-case on search criteria, this solution will easily become a bottle-neck for your system. As a result, your system may not scale well.
To best of my knowledge, I can suggest a few options that you may choose based on your requirement/budget to address this problem using a combination of databases.
Option 1. DynamoDB as a primary store and AWS Elasticsearch as secondary storage [Preferred]
Store the user records in DynamoDB table(let's call it UserTable)as and when a user registers.
Enable DynamoDB table streams on UserTable table.
Build an AWS Lambda function that reads from the table's stream and persists the records in AWS Elasticsearch.
Now in your application, use DynamoDB for fetching user records from id. For all other search criteria(like searching on emailId, phone number, zip code, location etc) fetch the records from AWS Elasticsearch. AWS Elasticsearch by default indexes all the attributes of your record, so you can search on any field within millisecond of latency.
Option 2. Use AWS Aurora [Less preferred solution]
If your application has a relational use-case where data are related, you may consider this option. Just to call out, Aurora is a SQL database.
Since this is a relational storage, you can opt for organizing the records in multiple tables and join them based on the primary key of those tables.
I will suggest for 1st option as:
DynamoDB will provide you durable, highly available, low latency primary storage for your application.
AWS Elasticsearch will act as secondary storage, which is also durable, scalable and low latency storage.
With AWS Elasticsearch, you can run any search query on your table. You can also do analytics on data. Kibana UI is provided out of the box, that you may use to plot the analytical data on a dashboard like (how user growth is trending, how many users belong to a specific location, user distribution based on city/state/country etc)
With DynamoDB streams and AWS Lambda, you will be syncing these two databases in near real-time [within few milliseconds]
Your application will be scalable and the search feature can further be enhanced to do filtering on multi-level attributes. [One such example: search all users who belong to a given city]
Having said that, now I will leave this up to you to decide. 😊