DynamoDB table design for social network - amazon-web-services

Question

I got a thinking-problem in DynamoDB.
My structure is looking as following:
primary key = "id"
sort key = "sort"
I have posts, users and "user A following user B" relationships.
Users:
id=1234
sort="USER_USER_1234"
name="max" (for example)
-
id=3245
sort="USER_USER_3245"
name="tom"
Post:
id=9874
sort="POST_POST_1234 (because its created by user id 1234)
createdAt=1560371687
Following:
id=1234
sort="USER_FOLLOW_3245"
--> tom follows max (but max not tom)
How could I design a query to get all posts by the people which tom(id=3245) is following? So in my case the post id 9874?
My approach was to put a GSI where sort is the primary key and id is the sort key (that i can query all people which user A is following), than get all the posts from the users (with help of the same GSI) and sort the result after a second index where createdAt is the sort key. The problem is that this needs much much querys (imagine user A would follow 10000 people and they all make posts). Is there a technique or design thinking approach which you could recommend for this situation? My second approach was to index the whole application table to elastic search and do a nested query. Would this make more sense? Or would you recommend using another type of database like AWS neptune?

Answer 1

There's a hands-on lab on aws about a similar problem - "a mobile application that includes a social network": https://aws.amazon.com/getting-started/hands-on/design-a-database-for-a-mobile-app-with-dynamodb/4/
Brief description:
Users will upload photos through your application
users will want to find and follow friends
By following a friend, a user will receive notifications of the friend’s new photos
user will be able to message their friends
friends can view their photos
users can react to a photo with one of four emojis — a heart, a smiley face, a thumbs up, or a pair of sunglasses.
When looking at a photo, users should be able to see the number of each type of reaction a photo has received
The model has the following entities: User, Photo, Reaction, Friendship.
A User can have many Photos, and a Photo can have many Reactions. Finally, the Friendship entity represents a many-to-many relationship between Users, as a User can follow multiple Users and be followed by multiple other Users.
Access patterns
Based on the business requirements, these are the access patterns identified:
User
Create user profile (Write)
Update user profile (Write)
Get user profile (Read)
Photo
Upload photo for user (Write)
View recent photos for user (Read)
React to a photo (Write)
View photo and reactions (Read)
Friendship
Users can follow friends, view updates on their friends’ activities, and receive recommendations on other friends they may want to follow.
A friendship is a one-way relationship, like Twitter. One user can choose to follow another user, and that user may choose to follow the user back. For our application, we will call the users that follow a user “followers”, and we will call the users that a user is following the “followed”.
Based on this information, we have the following access patterns:
Follow user (Write)
View followers for user (Read)
View followed for user (Read)
On the Friendship entity, we have an access pattern that needs to find all users that follow a particular user as well as an access pattern to find all of the users that a given user follows.
Table Design
Because of this, we’ll use a composite primary key with both a PK and SK value. The composite primary key will give us the Query ability on the PK to satisfy one of the query patterns we need:
Entity PK SK
User USER#<USERNAME> #METADATA#<USERNAME>
Photo USER#<USERNAME>. PHOTO#<USERNAME>#<TIMESTAMP>
Reaction REACTION#<USERNAME>#<TYPE> PHOTO#<USERNAME>#<TIMESTAMP>
Friendship USER#<USERNAME> #FRIEND#<FRIEND_USERNAME>
The Friendship entity uses the same PK as the User entity. This will allow you to fetch both the metadata for a user plus all of the user’s followers in a single query:
KeyConditionExpression="PK = :pk AND SK BETWEEN :metadata AND :photos",
ExpressionAttributeValues={
":pk": { "S": "USER#{}".format(username) },
":metadata": { "S": "#METADATA#{}".format(username) },
":photos": { "S": "PHOTO$" },
},
A secondary (inverted) index is useful to query the “other” side of a many-to-many relationship. This is the case for your Friendship entity. With your primary key structure, you can query all followers for a particular user with a query against the table’s primary key. When you add an inverted index, you will be able to find the users that a user is following (the “followed”) by querying the inverted index:
KeyConditionExpression="SK = :sk",
ExpressionAttributeValues={
":sk": { "S": "#FRIEND#{}".format(username) }
},
Extensions
What would be interesting is to tweak the design to support mega-popular users (having millions of followers).
Another interesting access pattern not mentioned here is the user feed - see all the photos that their friends have recently posted. This could be done with another table to contain this stream of data which gets updated whenever a friend posts something (find his followers, update their feeds...).

Answer 2

In Amazon Neptune, this would be something as simple as:
g.V(3245).E('post')
The above query would return an iterator, to all the vertices connected by the Edge label "post", starting from the vertex with ID "3245". You can firther tighten it up by either projecting specific properties (.property('name')) from those vertices or materializing the whole vertex (.valueMap()). This is just Gremlin syntax, and you can easily do the same using SPARQL as well, and Amazon Neptune supports both of them.
A bigger question for you is to evaluate all the types of queries you wish to perform on your data, and see if modeling it in a graph database makes sense. If it does, then you're better off using Neptune as opposed to something custom using a mix of other products. Querying/Traversing highly connected data, navigating through relationships etc are some of the classic usecases to use a graph data model.