I am making a blog website and I am facing some issues with the Query performance.
I have 3 models
User Model -> Users (To store user email, Password etc)
Post Model -> Actual Posts
people Model -> (To store users extra information)
Post Model ->
class Post(models.Model):
user = models.ForeignKey(User, on_delete=models.PROTECT)
category = models.ForeignKey(Category, on_delete=models.PROTECT)
title = models.CharField(max_length=255,null=True)
description = models.CharField(max_length=1000,null=True)
Likes = models.ManyToManyField(to=User, related_name='Post_likes')
favourites = models.ManyToManyField(to=User,blank=True,related_name="favourite")
People Model ->
class People(models.Model):
user = models.OneToOneField(User, on_delete=models.CASCADE)
photo = models.ImageField(upload_to='profile_pics', blank=True,null=True)
Phone_number = models.CharField(max_length=255,null=True,blank=True)
Birth_Date = models.DateField(null=True,blank=True)
Created_date = models.DateTimeField(auto_now_add=True)
Updated_date = models.DateTimeField(auto_now=True)
Now as both of these models are connected to User model. I want to query the Post model and get the user photo in the template. Now when I use post.user.people.photo then for every post it generates a seperate query to DB resulting in slowness. I would like to use Join here to combines multiple tables and fetch all the records at once.
I am currently using following Query ->
posts = Post.objects.select_related().prefetch_related('images_set').annotate(comments_Count = Count('comments_post',distinct=True)).annotate(Count('Likes',distinct=True)).all().order_by('-id')
You can perform a .select_related(…) [Django-doc] on the user and the people with user__people, so:
posts = Post.objects.select_related(
'user__people', 'category'
).prefetch_related('images_set').annotate(
comments_Count = Count('comments_post',distinct=True),
Count('Likes',distinct=True)
).order_by('-id')
Note: It is normally better to make use of the settings.AUTH_USER_MODEL [Django-doc] to refer to the user model, than to use the User model [Django-doc] directly. For more information you can see the referencing the User model section of the documentation.
Related
I am attempting to merge and pull data from three Django models into a view. Players and Events relate to each other in the Details model (a player can attend many events) using models.ForeignKey.
In other platforms I would have written a DB View to join tables and the application would query that view.
From what I understand Django does not support data views within Models.
Looking for help on how I would approach this in Django.
class Players(models.Model):
firstName = models.CharField(max_length=255)
lastName = models.CharField(max_length=255)
highSchool = models.CharField(max_length=255)
gradYear = models.IntegerField()
slug = models.SlugField(default="", null=False)
class Events(models.Model):
title = models.CharField(max_length=255)
location = models.CharField(max_length=255)
date = models.DateField()
class Details(models.Model):
event = models.ForeignKey(Events, on_delete=models.CASCADE)
player = models.ForeignKey(Players, on_delete=models.CASCADE)
height = models.IntegerField(default=None, blank=True)
weight = models.IntegerField(default=None, blank=True)
def playerdetail(request,slug):
playerinfo = Details.objects.get(id=1)
template = loader.get_template('playerdetail.html')
context = {
'playerinfo': playerinfo,
}
return HttpResponse(template.render(context, request))
You are actually doing what you needed to do with the code you provided.
When you are invoking a query on a model that connects those two entities (Players,Events), it performs the join when you try to access each of these properties through the foreign key fields.
For example, for accessing the player information (which makes the Django ORM perform the join operation in the background):
# Get the first name of the player
first_name = playerinfo.player.firstName
For filtering and showing in other places, you can use the notation field__subfield
For more information, please read the examples of this website:
https://books.agiliq.com/projects/django-orm-cookbook/en/latest/index.html
I am creating a Blog website where for each blog a user can like or dislike the post.
Now every time user goes on Index page I want to change the like button i.e if user has liked the post then a dislike button otherwise like button.
For this I need to get the variable IsLiked from the views.py file.
This is the Query that I have written.
posts = Post.objects.exclude(users=request.user).select_related('user__people','ProductAvailability').prefetch_related('images_set').annotate(comments_Count = Count('comments_post',distinct=True)).annotate(Count('Likes',distinct=True)).all().order_by('-id')
for post in posts:
if(post.Likes.filter(id=user_id).exists()):
isLiked = True
else:
isLiked = False
Here the problem is that for every post there is a separate query sent to DB.
This is my Blog Post Model ->
class Post(models.Model):
user = models.ForeignKey(User, on_delete=models.PROTECT)
# category = models.ForeignKey(Category, on_delete=models.PROTECT)
ProductAvailability = models.ForeignKey(ProductAvailability, on_delete=models.PROTECT, null=True, blank=True)
title = models.CharField(max_length=255,null=True)
description = models.CharField(max_length=1000,null=True)
Likes = models.ManyToManyField(to=User, related_name='Post_likes')
favourites = models.ManyToManyField(to=User,blank=True,related_name="favourite")
Tag1 = models.CharField(max_length=255,null=True,blank=True)
Tag2 = models.CharField(max_length=255,null=True,blank=True)
Tag3 = models.CharField(max_length = 255, null = True, blank = True)
Tag1_Name = models.CharField(max_length=255,null=True,blank=True)
Tag2_Name = models.CharField(max_length=255,null=True,blank=True)
Tag3_Name = models.CharField(max_length=255,null=True,blank=True)
users = models.ManyToManyField(User, related_name='users_hidden_from_post')
Created_date = models.DateTimeField(auto_now_add=True)
Updated_date = models.DateTimeField(auto_now=True)
PS: Please Ignore the redundant Info in the Post Model
I want to send the User id with the Query and check if individual post is liked by the user or not.
You can annotate the Posts with a condition is_liked that checks if such User appears in the through model of the likes of the Post with an Exists subquery [Django-doc]:
from django.db.models import Exists, OuterRef
posts = Post.objects.exclude(
users=request.user
).select_related(
'user__people', 'ProductAvailability'
).prefetch_related(
'images_set'
).annotate(
comments_Count = Count('comments_post',distinct=True),
Count('Likes',distinct=True),
is_liked=Exists(
Post.Likes.through.objects.filter(
post_id=OuterRef('pk'), user_id=user_id
)
)
).order_by('-id')
The Post objects that arise from this queryset will have an extra attribute named .is_liked that is True if the user_id appears in the Likes for that Post.
Note: It is normally better to make use of the settings.AUTH_USER_MODEL [Django-doc] to refer to the user model, than to use the User model [Django-doc] directly. For more information you can see the referencing the User model section of the documentation.
Note: normally the name of the fields in a Django model are written in snake_case, not PascalCase, so it should be: likes instead of Likes.
I'm working on a django website and I need to store some information about the user like a isVerified BooleanField and a profile picture in an ImageField as well as ratings which stores ratings a user has given different elements. So I made a model like this:
class UserProfile(AbstractBaseUser):
is_verified = models.BooleanField(default=True)
current_profile = models.ImageField(default=static('img/default_profile.jpg'))
ratings = models.ManyToManyField(Element, on_delete=models.CASCADE)
however I'd like to save some more about these ratings (like a timestamp and the actual value the user rated)
Do I need to make a seperate model just for that or can this be acchieved in a better way?
You need to use intermediary table that can be specified via the through keyword.
For example:
class UserProfile(AbstractBaseUser):
is_verified = models.BooleanField(default=True)
current_profile = models.ImageField(default=static('img/default_profile.jpg'))
ratings = models.ManyToManyField(Element, on_delete=models.CASCADE, through='UserProfileElement')
class UserProfileElement(models.Model):
user = models.ForeignKey(UserProfile, models.CASCADE, verbose_name=_('User'))
element = models.ForeignKey(Element, models.CASCADE, verbose_name=_('Element'))
timestamp = models.DateTimeField(_('Timestamp'), auto_now_add=True)
rating = models.PositiveIntegerField(_('Rating'))
class Meta:
unique_together = ('user', 'element')
Django docs: ManyToManyField.through
I am struggling to understand how one-to-many and many-to-many relation works in Django model. My schema looks something like this so far, I am open for suggestions to make it better.
A many-to-many relation between users and team. Also, there will be schedules that belong to a particular user of a team.
This is how my model looks like so far,
class Team(models.Model):
tid = models.AutoField(primary_key=True)
team_name = models.CharField(max_length=30)
manager_name = models.CharField(max_length=30)
class Schedule(models.Model):
sid = models.AutoField(primary_key=True)
user = models.OneToOneField(User)
date = models.DateField()
start_time = models.TimeField()
end_time = models.TimeField()
pay_rate = models.CharField(max_length=30)
location = models.CharField(max_length=50)
class BelongsTo(models.Model):
bid = models.AutoField(primary_key=True)
user = models.OneToOneField(User)
team = models.ForeignKey(Team, on_delete=models.CASCADE)
schedule = models.ForeignKey(Schedule, on_delete=models.CASCADE)
Question: I want to get the information of each user, what are their schedules and which team each schedule belongs to. How would I to do it? I have tried BelongsTo.objects.select_related().all(), but it is not working for me.
Note: I am open for suggestions, if something is wrong with my schema or model or the approach, please let me know.
BelongsTo is seems like utility table.So
BelongsTo.objects.all().values('user', 'team__team_name', 'schedule')
Your schema looks almost right, but I would modify it a little bit. In particular, I will change how Schedule is implemented. Instead of having a sid in the User Belongs To join-table, I would include the user and team in the Schedule table as foreign keys.
This is how the Django models should then look like:
class User(models.Model):
username = models.CharField(max_length = 200)
# put other fields like password etc. here
class Team(models.Model):
team_name = models.CharField(max_length=30)
manager_name = models.CharField(max_length=30)
user = models.ManyToManyField("User")
class Schedule(models.Model):
user = models.ForeignKey("User")
team = models.ForeignKey("Team")
date = models.DateField()
start_time = models.TimeField()
end_time = models.TimeField()
pay_rate = models.CharField(max_length=30)
location = models.CharField(max_length=50)
Note the following:
You don't need to have a primary key field in the models because Django adds a primary key field called pk or id automatically.
Note the absence of the User Belongs To model. Django implements join-tables like User Belongs To automatically when you use a ManyToManyField. See the Django docs on many-to-many relationships.
You also don't need on_delete = models.CASCADE on ForeignKey fields, because this is the default behavior.
To see how to get information about users, teams and schedule from this configuration of models, consult the Django documentation on making db queries. It's quite easy.
I have a fairly simple Django set up for a forum, and one of the most basic models is this, for each thread:
class Post(models.Model):
created = models.DateTimeField(auto_now_add=True)
last_reply = models.DateTimeField(auto_now_add=True, blank=True, null=True)
username = models.ForeignKey(User, related_name="forumuser")
fixed = models.BooleanField(_("Sticky"), default=False)
closed = models.BooleanField(default=False)
markdown_enabled = models.BooleanField(default=False)
reply_count = models.IntegerField(default=0)
title = models.CharField(_("Title Post"), max_length=255)
content = models.TextField(_("Content"), blank=False)
rating = models.IntegerField(default=0)
followers = models.IntegerField(default=0)
ip_address = models.CharField(max_length=255)
def __unicode__(self):
return self.title
def get_absolute_url(self):
return "/post/%s/" % self.id
Then we have some replies:
class PostReply(models.Model):
user = models.ForeignKey(User, related_name='replyuser')
post = models.ForeignKey(Post, related_name='replypost')
created = models.DateTimeField(auto_now_add=True)
content = models.TextField()
ip_address = models.CharField(max_length=255)
quoted_post = models.ForeignKey('self', related_name='quotedreply', blank=True, null=True)
rating = models.IntegerField(default=0)
reply_order = models.IntegerField(default=1)
Now, currently there just over 1600 users, 6000 Posts, and 330,000 PostReply objects in the db for this setup. When I run this SQL query:
SELECT * FROM `forum_post` LIMIT 10000
I see that Query took 0.0241 sec which is fine. When I browse to the Django admin section of my site, pulling up an individual Post is rapid, as is the paginated list of Posts.
However, if I try and pull up an individual PostReply, it takes around 2-3 minutes to load.
Obviously each PostReply admin page will have a dropdown list of all the Posts in it, but can anyone tell me why this or anything else would cause such a dramatically slow query? It's worth noting that the forum itself is pretty fast.
Also, if it is something to do with that dropdown list, has anyone got any suggestions for making that more usable?
Try to add all foreign keys in raw_id_fields in admin
class PostReplyAdmin(ModelAdmin):
raw_id_fields = ['user', 'post', 'quoted_post']
This will decrease page's load time in change view. The problem is that django loads ForeignModel.objects.all() for each foreign key's dropdowns.
Another way is to add foreign keys in autocomplete_fields (docs) in admin
class PostReplyAdmin(ModelAdmin):
autocomplete_fields = ['user', 'post', 'quoted_post']
As pointed by #Andrey Nelubin the problem for me was indeed in the page loading all related models for each foreign key's dropdown. However, with autocomplete_fields selects are turned into autocomplete inputs (see figure below), which load options asynchronously.