I'm new to DynamoDb and generally new to NoSql databases.
I'm currently playing arround with a simple webapplication which is written in asp.net core 6 an run on aws lambda.
I have posts which can be in different channels. Users can make comments on posts. Posts and Comments can be up/down voted.
When trying to implement voting on comments I started getting the feeling,
that my design is quite bad after reading for example the following question:
Update nested map dynamodb
The issue is that:
I can't update votes of a comment without knowing the array index
the array index is prone to race conditions when an older comment (with lower index therefore) is deleted
Current data structur (3 tables):
guids are saved as strings
post: {
pk -> id
sk -> id
id : guid
createdAt : date
createdBy : guid // user.id
comments: [
{
id : guid
text : string
createdAt : date
createdBy : guid // user.id
upvotedBy : guid[] // user ids
downvotedBy : guid[] // user ids
upvotes : int // aggregated count of above list
downvotes : int
},
{ ... },
{ ... },
{ ... },
]
commentCount : int
upvotedBy : guid[] // user ids
downvotedBy : guid[] // user ids
upvotes : int // aggregated count of above list
downvotes : int
channel : guid // channel.id
hidden : bool
}
user: {
pk -> id
sk -> id
id : guid
username : string
password : string
createdAt : date
subscribed_channels : guid[] // channel ids
}
channel: {
pk -> id
sk -> id
id : guid / string
channelname : string
}
I have additional keys for:
- posts
- ChannelIndex : pk = channelname, sk = createdAt
- CreatedAtIndex : pk = pk, sk = createdAt
- users
- UsernameIndex : pk = username, sk = sk
- channels
- ChannelnameIndex : pk = channelname, sk = sk
Usecases (ordered by frequency):
Access all posts without comments
from multiple channels
from single channels
sorted by date
sorted by most comments
sorted by most likes
from all channels
Access single post with (all) comments
vote on comment
vote on post
create comment
create post
delete comment
delete post
Question:
How to improve this design from a performance but also from the ease of implementation side?
My ideas:
I thought about removing the nesting for comments making them a own table with a key beeing postId and createdAt
However this feels like the design for a relational database.
Related
class A:
some fields
class B:
user = models.ForeignKey(User, **CASCADE, related_name='sessions')
DEVICES = (
('android', 'android'),
('ios', 'ios')
)
We are creating a dashboard for front end and there is a search field, user supposed to type the device_type and we used to fetch the records who uses only the IOS or Android.
In this Case the records is stored in the fields like,
Created_at, updated_at, device_type, user_id...
we need to apply two filters now here:
searched_key : ios or android
latest_device_type : only latest updated_record
I have done like :
User.objects.filter(Q(id=user_details['id']) & Q(sessions__device_type=self.search_key) & Q(reduce(or_, self.query_filter))).order_by('sessions__user_id', '-sessions__created_at').distinct('sessions__user_id'). \
values('id', 'sessions__device_type', 'first_name')
But am not getting exact output :
even i tried [0] putting this after order_by and latest() and first() method will fetch only one record so any 1 has the best solution for this.
Note : if anybody try to explain with Prefetch please explain with clarity.
I am all new to NoSQL and specifically DynamoDB single table design. Have been going through a lot of videos and articles on the internet regarding the single-table design and finally I have put together a small design for a chat application which I am planning to build in the future.
The access patterns I have so far thought about are -
Get user details by User Id.
Get list of conversations the user is part of.
Get list of messages the user has created
Get all members of a conversation
Get all messages of a conversation
Also want to access messages of a conversation by a date range, so far I haven't figured out that one.
As per the below design, if I were to pull all messages of a conversation, is that going to pull the actual message in the message attribute which is in the message partition?
Here is the snip of the model I have created with some sample data on. Please let me know if I am in the right direction.
As per the below design, if I were to pull all messages of a conversation, is that going to pull the actual message in the message attribute which is in the message partition?
No, it will only return the IDs of a message as the actual content is in a separate partition.
I'd propose a different model - it consists of a table with a Global Secondary Indexe (GSI1). The layout is like this:
Base Table:
Partition Key: PK
Sort Key: SK
Global Secondary Index GSI1:
Partition Key: GSI1PK
Sort Key: GSI1SK
Base Table
GSI 1
Access Patterns
1.) Get user details by User Id.
GetItem on Base Table with Partition Key = PK = U#<id> and Sort Key SK = USER
2.) Get list of conversations the user is part of.
Query on Base Table with Partition Key = PK = U#<id> and Sort Key SK = starts_with(CONV#)
3.) Get list of messages the user has created
Query on GSI1 with Partition Key GSI1PK = U#<id>
4.) Get all members of a conversation
Query on Base Table with Partition Key = PK = CONV#<id> and Sort Key SK starts_with(U#)
5.) Get all messages of a conversation
Query on Base Table with Partition Key PK = CONV#<id> and Sort Key SK starts_with(MSG#)
6.) Also want to access messages of a conversation by a date range, so far I haven't figured out that one.
DynamoDB does Byte-Order Sorting in a partition - if you format all dates according to ISO 8601 in the UTC timezone, you can make the range query, e.g.:
Query on Base Table with Partition Key PK = CONV#<id> and Sort Key SK between(MSG#2021-09-20, MSG#2021-09-30)
I have one to many relationship table. this is my model
class Pointofsale(models.Model):
official_receipt = models.CharField(max_length=250,blank=False,null=False)
transaction_number = models.CharField(max_length=250,unique=True)
company = models.ForeignKey("User_company")
customer = models.ForeignKey("Customer")
class Pointofsale_details(models.Model):
pointofsale = models.ForeignKey(Pointofsale, related_name='pointofsale_details')
item = models.ForeignKey("Inventory")
quantity = models.DecimalField(max_digits = 19, decimal_places = 6, default = 0.0)
I want to get the sales with the items in this format
{ official_receipt :100,..,etc, pointofsale_details : [ {item1} , {item2} ] }
OR ( for summary reporting purposes )
[{ official_receipt :100,..,etc, pointofsale_details : [ {item1} , {item2} ] },{ official_receipt :100,..,etc, pointofsale_details : [ {item1} , {item2} ] }]
I tried this query for a single query but to no avail:
transaction_details = Pointofsale.objects.filter(company=get_current_company(request)).prefetch_related('pointofsale_details')
I get this error
'exceptions.NameError' object does not support item assignment
Is this is correct or proper way to handle one to many relationships?
What am I missing?
If this is unclear please feel free to comment below, so i could refine my question. Thank you!
Cheers!
def get_current_company(request,company_obj = False):
company = request.session.get('company_id',None)
return company
References:
Django query in One to Many relationship
How to select many to one to many without hundreds of queries using Django ORM?
Django ORM: Selecting related set
Right now i'm study a flow of ecommerce site using laravel 5.0 and crinsane laravel package .
I have setup 2 tables
Which is transactions and orders table
The relations is orders has many transactions (1 transaction 1 type of item ) , and transactions belong to orders .
So , in transactions there is foreign key order_id which references to order tables id .
In routes I set route::post('checkout','OrderController#checkoutpost');
public function checkoutpost()
{
// Get input from checkout forms
$input = Request::all();
// Insert forms data into Order table
Order::create($input);
// Retrieve the session data and inserting into Transaction table
$formid = str_random();
$cart_content = Cart::content();
foreach ($cart_content as $cart) {
$transaction = new Transaction();
$products = Product::find($cart->id);
$transaction->product_id = $cart->id;
$transaction->form_id = $formid;
$transaction->qty = $cart->qty;
$transaction->total_price = $cart->price * $cart->qty;
// Here is the problem , how to assign this transaction>order_id into our "id" that just inserted earlier ..
$transaction->order_id = $orders;
$transaction->save();
Cart::destroy();
return redirect('product/checkout');
}
}
The problem is how to assign order_id with the id of data that we just insert earlier?
Any feedback were really appreciated, thank you
Firstly, when creating the Order you need to assign the return value:
// An instance of Order is returned, so the id is accessible.
$order = Order::create($input);
Then you can use:
// Remember to make 'id' a fillable field on the Order model if you want to do it this way.
$transaction->order_id = $order->id;
Have you try this AvoRed an Laravel E commerce its almost fully featured e commerce for Laravel if you like it give it a try and let me know the feedback if you have any.
AvoRed An Laravel E commerce
I'm using servicestack and i'm planning to use ormlite for the data access layer.
I've these tables (SQL Server 2005)
Table ITEM
ID PK
...
Table SUBITEM1
ID PK FK -> ITEM(ID)
...
Table SUBITEM2
ID PK FK -> ITEM(ID)
...
Table POST
ID PK
...
Table COMMENT
ID PK
...
Table DOWNLOAD
ID PK
...
Table POST_COMMENT
ID PK
POST_ID FK -> POST(ID)
COMMENT_ID FK -> COMMENT(ID)
Table DOWNLOAD_COMMENT
ID PK
DOWNLOAD_ID FK -> DOWNLOAD(ID)
COMMENT_ID FK -> COMMENT(ID)
I created a class for each table and mapped them using annotations (autoincrement, references, etc).
I decided to creare a repository for each "entity" (item, post, comment, download).
Each repository contains the basic CRUD logic,
eg. 1 CommentRepository has a Save(Comment comment, Object relationship) that performs db.Insert(comment, relationship) where relationship is PostComment or DownloadComment.
eg. 2 PostRepository has a Save(Post p) that performs the insert into POST.
I'm not sure about this solution because the repository interface is different and I can't do polymorphic queries.
Can you provide any advice to improve my DAL?
Thank you for your attention.
I'm not a fan of forced artificial abstraction so I don't like starting with a repository for every entity as it will just lead to un-necessary code-bloat. I like to start with only 1 repository for all entities encapsulating all data access and refactor naturally out when it gets too big.
I don't know enough of your domain to know what the best RDBMS layout is but I also like to avoid creating un-necessary tables where possible and will look to blob non-aggregate root data, e.g. if SubItem can only applies and is not meaningful outside the context of its parent Item, then I would just blob it saving a couple of tables, e.g:
class Item {
int Id; //PK, AutoIncr
List<SubItem> SubItem;
}
Rather than the separate Many : Many tables, I would just maintain it on the single Comment table, e.g:
class Comment {
int Id; //PK, AutoIncr
string CommentType; //i.e. Post or Download
int RefId;
string Comment;
}
So my repository would mimic the data access patterns required for fulfilling the Web Request, something like:
class BlogRepository {
void AddCommentToPost(int postId, Comment comment);
void AddCommentToDownload(int downloadId, Comment comment);
}